2021-02-20T12:58:15.278 INFO:root:teuthology version: 1.1.0-9a6b849c 2021-02-20T12:58:15.284 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name trociny-2021-02-20_12:31:02-rbd-wip-mgolub-testing-distro-basic-smithi --description rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} --owner scheduled_trociny@teuthology --archive /home/teuthworker/archive/trociny-2021-02-20_12:31:02-rbd-wip-mgolub-testing-distro-basic-smithi/5898608 -- /home/teuthworker/archive/trociny-2021-02-20_12:31:02-rbd-wip-mgolub-testing-distro-basic-smithi/5898608/orig.config.yaml 2021-02-20T12:58:15.365 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-02-20T12:58:15.443 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/trociny-2021-02-20_12:31:02-rbd-wip-mgolub-testing-distro-basic-smithi/5898608 branch: wip-mgolub-testing description: rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} email: to.my.trociny@gmail.com first_in_suite: false job_id: '5898608' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: run two ceph clusters and install rbd-mirror - desc: configure the permissions for client.mirror - desc: 2 ceph clusters with 1 mon and 3 osds each - desc: run four rbd-mirror daemons per cluster - desc: run the rbd_mirror_stress.sh workunit to test the rbd-mirror daemon name: trociny-2021-02-20_12:31:02-rbd-wip-mgolub-testing-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 3 size: 30 os_type: ubuntu os_version: '18.04' overrides: admin_socket: branch: wip-mgolub-testing ceph: conf: client: debug rbd: 20 debug rbd_mirror: 15 log to stderr: false rbd default features: 125 rbd mirror image policy type: none client.mirror.0: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.1: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.2: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.3: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid global: mon client directed command retry: 5 ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: lz4 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs log-ignorelist: - \(OSD_SLOW_PING_TIME log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 0d796331675b92762d821b40e70f6598bcea1178 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: 0d796331675b92762d821b40e70f6598bcea1178 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mgolub-testing sha1: 0d796331675b92762d821b40e70f6598bcea1178 owner: scheduled_trociny@teuthology priority: 50 repo: https://github.com/ceph/ceph-ci.git roles: - - cluster1.mon.a - cluster1.mgr.x - cluster2.mgr.x - cluster1.osd.0 - cluster1.osd.1 - cluster1.osd.2 - cluster1.client.0 - cluster2.client.0 - - cluster2.mon.a - cluster2.osd.0 - cluster2.osd.1 - cluster2.osd.2 - cluster1.client.mirror - cluster1.client.mirror.0 - cluster1.client.mirror.1 - cluster1.client.mirror.2 - cluster1.client.mirror.3 - cluster1.client.mirror.4 - cluster1.client.mirror.5 - cluster1.client.mirror.6 - cluster2.client.mirror - cluster2.client.mirror.0 - cluster2.client.mirror.1 - cluster2.client.mirror.2 - cluster2.client.mirror.3 - cluster2.client.mirror.4 - cluster2.client.mirror.5 - cluster2.client.mirror.6 sha1: 0d796331675b92762d821b40e70f6598bcea1178 sleep_before_teardown: 0 suite: rbd suite_branch: wip-mgolub-testing suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_0d796331675b92762d821b40e70f6598bcea1178/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 0d796331675b92762d821b40e70f6598bcea1178 targets: smithi170.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDfrY9y6okoGGmSdGP0lWbQ4ocf5wj8XLxLWzIzKeOEY6RLGsDGfOqyKmBHiiztSFrBhM9jjTJxOvvPlTSrU9d2Dgm1QmuEjYA1JLPSaO8piMG/OlaSjI4PHxt9nxvTu77gXoXfXpK6Qo+ZUqZRv3OF9nApXNfTuDTYec8solgHl1hrUrW+n+Ow9gmRFNrBkDqVvEnYG7Xg+EQEm5aelmrdIhIdLh5Mjum14tIxk4i/1a2umeJOt88yzRzjLS2+H2/EXUn5QOGGpkz6mCS1Xd5L9UP4JVeYG9wdykGBhPJykA6OCzH5JEbGUEvqoSruNpN65AWm8PTlQ9hYB4OoK0VX smithi183.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC7bsnX0wXavEUXQ1xu3LEdpYndhq6jmv7ggYLSbXPw77TjZBIbrBrtxwh0XeWqiP9d2H0n0hf/drGz5LjqiplvzH9leTtkoAE3Y+0niWLqunqLFKlAqLRKPSrZKtJtc0kDrn34lpO5aORxGAD/ZfqsRrXLLNYgD3VV9XLNPPldmEFTjPOXJa1R2mAhDDfEFXuj/pZxhL+LlJytgvOI6EGcrYgixGtDVoQ2b8zbsz/4Z/0YN/TvhEE+TZs22LKfPiivEoAMv21FVua6WDv9WX526ChOhnkLnBFJXRavZG4r14sd4+2aWAgd+cS47CDYTuJQxs2hdzY8crT3ZRJPsIRL tasks: - install: extra_packages: - rbd-mirror - ceph: cluster: cluster1 - ceph: cluster: cluster2 - exec: cluster1.client.mirror.0: - sudo ceph --cluster cluster1 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd' - rbd-mirror: client: cluster1.client.mirror.0 thrash: true - rbd-mirror: client: cluster1.client.mirror.1 thrash: true - rbd-mirror: client: cluster1.client.mirror.2 thrash: true - rbd-mirror: client: cluster1.client.mirror.3 thrash: true - rbd-mirror: client: cluster2.client.mirror.0 thrash: true - rbd-mirror: client: cluster2.client.mirror.1 thrash: true - rbd-mirror: client: cluster2.client.mirror.2 thrash: true - rbd-mirror: client: cluster2.client.mirror.3 thrash: true - rbd-mirror-thrash: cluster: cluster1 - rbd-mirror-thrash: cluster: cluster2 - workunit: clients: cluster1.client.mirror: - rbd/rbd_mirror_stress.sh env: CEPH_ARGS: '' MIRROR_IMAGE_MODE: snapshot MIRROR_POOL_MODE: image RBD_IMAGE_FEATURES: layering,exclusive-lock RBD_MIRROR_INSTANCES: '4' RBD_MIRROR_USE_EXISTING_CLUSTER: '1' RBD_MIRROR_USE_RBD_MIRROR: '1' timeout: 6h teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.29353 2021-02-20T12:58:15.444 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_0d796331675b92762d821b40e70f6598bcea1178/qa; will attempt to use it 2021-02-20T12:58:15.454 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_0d796331675b92762d821b40e70f6598bcea1178/qa/tasks 2021-02-20T12:58:15.454 INFO:teuthology.run_tasks:Running task internal.check_packages... 2021-02-20T12:58:15.457 INFO:teuthology.task.internal:Checking packages... 2021-02-20T12:58:15.544 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'basic' and ceph hash '0d796331675b92762d821b40e70f6598bcea1178' 2021-02-20T12:58:15.545 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-02-20T12:58:15.545 INFO:teuthology.packaging:ref: None 2021-02-20T12:58:15.546 INFO:teuthology.packaging:tag: None 2021-02-20T12:58:15.546 INFO:teuthology.packaging:branch: wip-mgolub-testing 2021-02-20T12:58:15.547 INFO:teuthology.packaging:sha1: 0d796331675b92762d821b40e70f6598bcea1178 2021-02-20T12:58:15.547 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=wip-mgolub-testing 2021-02-20T12:58:15.752 INFO:teuthology.task.internal:Found packages for ceph version 17.0.0-936-g0d796331-1bionic 2021-02-20T12:58:15.753 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2021-02-20T12:58:15.764 INFO:teuthology.task.internal:no buildpackages task found 2021-02-20T12:58:15.765 INFO:teuthology.run_tasks:Running task internal.save_config... 2021-02-20T12:58:15.779 INFO:teuthology.task.internal:Saving configuration 2021-02-20T12:58:15.805 INFO:teuthology.run_tasks:Running task internal.check_lock... 2021-02-20T12:58:15.816 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-20T12:58:15.835 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-02-20 12:52:56.970107', 'description': '/home/teuthworker/archive/trociny-2021-02-20_12:31:02-rbd-wip-mgolub-testing-distro-basic-smithi/5898608', 'locked_by': 'scheduled_trociny@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPRiE9+E7uT3qRdhQtVTXstqNV+5kwLOOhN3yD1vpz3A', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi170.front.sepia.ceph.com'} 2021-02-20T12:58:15.852 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-02-20 12:52:56.970128', 'description': '/home/teuthworker/archive/trociny-2021-02-20_12:31:02-rbd-wip-mgolub-testing-distro-basic-smithi/5898608', 'locked_by': 'scheduled_trociny@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGcZqrzkHK8PhKzosz6r95BhrCFYbxY2/x1du1MSdC7B', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi183.front.sepia.ceph.com'} 2021-02-20T12:58:15.852 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2021-02-20T12:58:15.868 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'] 2021-02-20T12:58:15.869 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6'] 2021-02-20T12:58:15.869 INFO:teuthology.run_tasks:Running task console_log... 2021-02-20T12:58:16.008 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f537eef9268>, signals=[15]) 2021-02-20T12:58:16.022 INFO:teuthology.run_tasks:Running task internal.connect... 2021-02-20T12:58:16.035 INFO:teuthology.task.internal:Opening connections... 2021-02-20T12:58:16.036 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2021-02-20T12:58:16.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T12:58:16.111 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2021-02-20T12:58:16.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T12:58:16.187 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2021-02-20T12:58:16.198 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2021-02-20T12:58:16.313 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2021-02-20T12:58:16.314 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2021-02-20T12:58:16.361 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="Ubuntu" 2021-02-20T12:58:16.362 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-02-20T12:58:16.362 INFO:teuthology.orchestra.run.smithi170.stdout:ID=ubuntu 2021-02-20T12:58:16.362 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE=debian 2021-02-20T12:58:16.362 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-02-20T12:58:16.363 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="18.04" 2021-02-20T12:58:16.363 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://www.ubuntu.com/" 2021-02-20T12:58:16.363 INFO:teuthology.orchestra.run.smithi170.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-02-20T12:58:16.363 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-02-20T12:58:16.363 INFO:teuthology.orchestra.run.smithi170.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-02-20T12:58:16.364 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_CODENAME=bionic 2021-02-20T12:58:16.364 INFO:teuthology.orchestra.run.smithi170.stdout:UBUNTU_CODENAME=bionic 2021-02-20T12:58:16.365 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2021-02-20T12:58:16.381 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2021-02-20T12:58:16.394 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2021-02-20T12:58:16.394 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2021-02-20T12:58:16.441 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="Ubuntu" 2021-02-20T12:58:16.441 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-02-20T12:58:16.441 INFO:teuthology.orchestra.run.smithi183.stdout:ID=ubuntu 2021-02-20T12:58:16.442 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE=debian 2021-02-20T12:58:16.442 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-02-20T12:58:16.442 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="18.04" 2021-02-20T12:58:16.442 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://www.ubuntu.com/" 2021-02-20T12:58:16.443 INFO:teuthology.orchestra.run.smithi183.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-02-20T12:58:16.443 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-02-20T12:58:16.443 INFO:teuthology.orchestra.run.smithi183.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-02-20T12:58:16.443 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_CODENAME=bionic 2021-02-20T12:58:16.443 INFO:teuthology.orchestra.run.smithi183.stdout:UBUNTU_CODENAME=bionic 2021-02-20T12:58:16.444 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2021-02-20T12:58:16.458 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2021-02-20T12:58:16.477 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2021-02-20T12:58:16.489 INFO:teuthology.task.internal:Checking for old test directory... 2021-02-20T12:58:16.490 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2021-02-20T12:58:16.493 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2021-02-20T12:58:16.499 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2021-02-20T12:58:16.509 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2021-02-20T12:58:16.510 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2021-02-20T12:58:16.544 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2021-02-20T12:58:16.561 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2021-02-20T12:58:16.610 INFO:teuthology.run_tasks:Running task kernel... 2021-02-20T12:58:16.643 DEBUG:teuthology.task.kernel:normalized config {'cluster1.mon.a': {'kdb': True, 'sha1': 'distro'}, 'cluster1.mgr.x': {'kdb': True, 'sha1': 'distro'}, 'cluster2.mgr.x': {'kdb': True, 'sha1': 'distro'}, 'cluster1.osd.0': {'kdb': True, 'sha1': 'distro'}, 'cluster1.osd.1': {'kdb': True, 'sha1': 'distro'}, 'cluster1.osd.2': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.0': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.0': {'kdb': True, 'sha1': 'distro'}, 'cluster2.mon.a': {'kdb': True, 'sha1': 'distro'}, 'cluster2.osd.0': {'kdb': True, 'sha1': 'distro'}, 'cluster2.osd.1': {'kdb': True, 'sha1': 'distro'}, 'cluster2.osd.2': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.0': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.1': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.2': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.3': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.4': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.5': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.6': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.0': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.1': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.2': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.3': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.4': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.5': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.6': {'kdb': True, 'sha1': 'distro'}} 2021-02-20T12:58:16.644 INFO:teuthology.task.kernel:config {'cluster1.mon.a': {'kdb': True, 'sha1': 'distro'}, 'cluster2.mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2021-02-20T12:58:16.646 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2021-02-20T12:58:16.655 INFO:teuthology.orchestra.run.smithi170.stdout:4.15.0-134-generic 2021-02-20T12:58:16.658 INFO:teuthology.task.kernel:Running kernel on smithi170: 4.15.0-134-generic 2021-02-20T12:58:16.658 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get clean 2021-02-20T12:58:16.748 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get update 2021-02-20T12:58:17.012 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-02-20T12:58:17.012 INFO:teuthology.orchestra.run.smithi170.stdout:Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2021-02-20T12:58:17.074 INFO:teuthology.orchestra.run.smithi170.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-02-20T12:58:17.343 INFO:teuthology.orchestra.run.smithi170.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-02-20T12:58:17.412 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [916 kB] 2021-02-20T12:58:17.541 INFO:teuthology.orchestra.run.smithi170.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,885 kB] 2021-02-20T12:58:17.798 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,544 kB] 2021-02-20T12:58:17.909 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [298 kB] 2021-02-20T12:58:17.927 INFO:teuthology.orchestra.run.smithi170.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [226 kB] 2021-02-20T12:58:17.936 INFO:teuthology.orchestra.run.smithi170.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [29.9 kB] 2021-02-20T12:58:17.937 INFO:teuthology.orchestra.run.smithi170.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [975 kB] 2021-02-20T12:58:17.941 INFO:teuthology.orchestra.run.smithi170.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,220 kB] 2021-02-20T12:58:17.971 INFO:teuthology.orchestra.run.smithi170.stdout:Get:13 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,109 kB] 2021-02-20T12:58:17.976 INFO:teuthology.orchestra.run.smithi170.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [390 kB] 2021-02-20T12:58:17.988 INFO:teuthology.orchestra.run.smithi170.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [247 kB] 2021-02-20T12:58:17.994 INFO:teuthology.orchestra.run.smithi170.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [33.4 kB] 2021-02-20T12:58:17.995 INFO:teuthology.orchestra.run.smithi170.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,560 kB] 2021-02-20T12:58:18.000 INFO:teuthology.orchestra.run.smithi170.stdout:Get:18 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [248 kB] 2021-02-20T12:58:18.061 INFO:teuthology.orchestra.run.smithi170.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,718 kB] 2021-02-20T12:58:18.112 INFO:teuthology.orchestra.run.smithi170.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [363 kB] 2021-02-20T12:58:18.116 INFO:teuthology.orchestra.run.smithi170.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [14.3 kB] 2021-02-20T12:58:18.116 INFO:teuthology.orchestra.run.smithi170.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [31.8 kB] 2021-02-20T12:58:18.117 INFO:teuthology.orchestra.run.smithi170.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [7,300 B] 2021-02-20T12:58:19.612 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 13.1 MB in 3s (5,121 kB/s) 2021-02-20T12:58:21.011 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2021-02-20T12:58:21.057 DEBUG:teuthology.orchestra.run.smithi170:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-02-20T12:58:21.139 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2021-02-20T12:58:21.390 INFO:teuthology.orchestra.run.smithi170.stdout:Building dependency tree... 2021-02-20T12:58:21.391 INFO:teuthology.orchestra.run.smithi170.stdout:Reading state information... 2021-02-20T12:58:21.470 INFO:teuthology.orchestra.run.smithi170.stderr:E: Unable to locate package linux-image-current-generic 2021-02-20T12:58:21.472 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-02-20T12:58:21.472 DEBUG:teuthology.orchestra.run.smithi170:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-02-20T12:58:21.552 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2021-02-20T12:58:21.783 INFO:teuthology.orchestra.run.smithi170.stdout:Building dependency tree... 2021-02-20T12:58:21.784 INFO:teuthology.orchestra.run.smithi170.stdout:Reading state information... 2021-02-20T12:58:21.960 INFO:teuthology.orchestra.run.smithi170.stdout:The following additional packages will be installed: 2021-02-20T12:58:21.961 INFO:teuthology.orchestra.run.smithi170.stdout: linux-generic linux-headers-4.15.0-135 linux-headers-4.15.0-135-generic 2021-02-20T12:58:21.961 INFO:teuthology.orchestra.run.smithi170.stdout: linux-headers-generic linux-image-4.15.0-135-generic 2021-02-20T12:58:21.961 INFO:teuthology.orchestra.run.smithi170.stdout: linux-modules-4.15.0-135-generic linux-modules-extra-4.15.0-135-generic 2021-02-20T12:58:21.963 INFO:teuthology.orchestra.run.smithi170.stdout:Suggested packages: 2021-02-20T12:58:21.963 INFO:teuthology.orchestra.run.smithi170.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2021-02-20T12:58:21.963 INFO:teuthology.orchestra.run.smithi170.stdout:Recommended packages: 2021-02-20T12:58:21.963 INFO:teuthology.orchestra.run.smithi170.stdout: thermald 2021-02-20T12:58:21.998 INFO:teuthology.orchestra.run.smithi170.stdout:The following NEW packages will be installed: 2021-02-20T12:58:21.998 INFO:teuthology.orchestra.run.smithi170.stdout: linux-headers-4.15.0-135 linux-headers-4.15.0-135-generic 2021-02-20T12:58:21.998 INFO:teuthology.orchestra.run.smithi170.stdout: linux-image-4.15.0-135-generic linux-modules-4.15.0-135-generic 2021-02-20T12:58:21.999 INFO:teuthology.orchestra.run.smithi170.stdout: linux-modules-extra-4.15.0-135-generic 2021-02-20T12:58:21.999 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages will be upgraded: 2021-02-20T12:58:22.000 INFO:teuthology.orchestra.run.smithi170.stdout: linux-generic linux-headers-generic linux-image-generic 2021-02-20T12:58:22.177 INFO:teuthology.orchestra.run.smithi170.stdout:3 upgraded, 5 newly installed, 0 to remove and 38 not upgraded. 2021-02-20T12:58:22.177 INFO:teuthology.orchestra.run.smithi170.stdout:Need to get 66.0 MB of archives. 2021-02-20T12:58:22.178 INFO:teuthology.orchestra.run.smithi170.stdout:After this operation, 334 MB of additional disk space will be used. 2021-02-20T12:58:22.178 INFO:teuthology.orchestra.run.smithi170.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-135-generic amd64 4.15.0-135.139 [13.1 MB] 2021-02-20T12:58:23.146 INFO:teuthology.orchestra.run.smithi170.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-135-generic amd64 4.15.0-135.139 [8,019 kB] 2021-02-20T12:58:23.341 INFO:teuthology.orchestra.run.smithi170.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-135-generic amd64 4.15.0-135.139 [32.8 MB] 2021-02-20T12:58:24.473 INFO:teuthology.orchestra.run.smithi170.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.135.122 [1,868 B] 2021-02-20T12:58:24.551 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.135.122 [2,592 B] 2021-02-20T12:58:24.630 INFO:teuthology.orchestra.run.smithi170.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-135 all 4.15.0-135.139 [10.9 MB] 2021-02-20T12:58:25.512 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-135-generic amd64 4.15.0-135.139 [1,097 kB] 2021-02-20T12:58:25.612 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.135.122 [2,488 B] 2021-02-20T12:58:26.079 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 66.0 MB in 4s (18.3 MB/s) 2021-02-20T12:58:26.243 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-modules-4.15.0-135-generic. 2021-02-20T12:58:26.282 INFO:teuthology.orchestra.run.smithi170.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102079 files and directories currently installed.) 2021-02-20T12:58:26.285 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../0-linux-modules-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T12:58:26.332 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-modules-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T12:58:27.967 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-image-4.15.0-135-generic. 2021-02-20T12:58:27.977 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../1-linux-image-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T12:58:28.078 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-image-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T12:58:28.602 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-135-generic. 2021-02-20T12:58:28.614 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T12:58:28.652 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-modules-extra-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T12:58:32.421 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../3-linux-generic_4.15.0.135.122_amd64.deb ... 2021-02-20T12:58:32.906 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-generic (4.15.0.135.122) over (4.15.0.134.121) ... 2021-02-20T12:58:33.212 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.135.122_amd64.deb ... 2021-02-20T12:58:33.316 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-image-generic (4.15.0.135.122) over (4.15.0.134.121) ... 2021-02-20T12:58:33.670 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-headers-4.15.0-135. 2021-02-20T12:58:33.683 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../5-linux-headers-4.15.0-135_4.15.0-135.139_all.deb ... 2021-02-20T12:58:33.720 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-headers-4.15.0-135 (4.15.0-135.139) ... 2021-02-20T12:58:38.332 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-headers-4.15.0-135-generic. 2021-02-20T12:58:38.355 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../6-linux-headers-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T12:58:38.382 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-headers-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T12:58:40.849 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.135.122_amd64.deb ... 2021-02-20T12:58:40.970 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-headers-generic (4.15.0.135.122) over (4.15.0.134.121) ... 2021-02-20T12:58:41.298 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-modules-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T12:58:41.851 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-headers-4.15.0-135 (4.15.0-135.139) ... 2021-02-20T12:58:41.977 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-image-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T12:58:42.418 INFO:teuthology.orchestra.run.smithi170.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-135-generic 2021-02-20T12:58:42.418 INFO:teuthology.orchestra.run.smithi170.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-135-generic 2021-02-20T12:58:42.546 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-modules-extra-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T12:58:43.231 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-headers-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T12:58:43.349 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-image-generic (4.15.0.135.122) ... 2021-02-20T12:58:43.469 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-headers-generic (4.15.0.135.122) ... 2021-02-20T12:58:43.592 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-generic (4.15.0.135.122) ... 2021-02-20T12:58:43.719 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for linux-image-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T12:58:43.939 INFO:teuthology.orchestra.run.smithi170.stdout:/etc/kernel/postinst.d/initramfs-tools: 2021-02-20T12:58:43.940 INFO:teuthology.orchestra.run.smithi170.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-135-generic 2021-02-20T12:58:58.821 INFO:teuthology.orchestra.run.smithi170.stdout:/etc/kernel/postinst.d/zz-update-grub: 2021-02-20T12:58:58.821 INFO:teuthology.orchestra.run.smithi170.stdout:Sourcing file `/etc/default/grub' 2021-02-20T12:58:58.851 INFO:teuthology.orchestra.run.smithi170.stdout:Generating grub configuration file ... 2021-02-20T12:58:59.231 INFO:teuthology.orchestra.run.smithi170.stdout:Found linux image: /boot/vmlinuz-4.15.0-135-generic 2021-02-20T12:58:59.264 INFO:teuthology.orchestra.run.smithi170.stdout:Found initrd image: /boot/initrd.img-4.15.0-135-generic 2021-02-20T12:58:59.637 INFO:teuthology.orchestra.run.smithi170.stdout:Found linux image: /boot/vmlinuz-4.15.0-134-generic 2021-02-20T12:58:59.667 INFO:teuthology.orchestra.run.smithi170.stdout:Found initrd image: /boot/initrd.img-4.15.0-134-generic 2021-02-20T12:59:00.496 INFO:teuthology.orchestra.run.smithi170.stdout:done 2021-02-20T12:59:00.909 DEBUG:teuthology.orchestra.run.smithi170:> dpkg -s linux-image-generic 2021-02-20T12:59:00.933 INFO:teuthology.orchestra.run.smithi170.stdout:Package: linux-image-generic 2021-02-20T12:59:00.933 INFO:teuthology.orchestra.run.smithi170.stdout:Status: install ok installed 2021-02-20T12:59:00.934 INFO:teuthology.orchestra.run.smithi170.stdout:Priority: optional 2021-02-20T12:59:00.934 INFO:teuthology.orchestra.run.smithi170.stdout:Section: kernel 2021-02-20T12:59:00.934 INFO:teuthology.orchestra.run.smithi170.stdout:Installed-Size: 17 2021-02-20T12:59:00.935 INFO:teuthology.orchestra.run.smithi170.stdout:Maintainer: Ubuntu Kernel Team 2021-02-20T12:59:00.935 INFO:teuthology.orchestra.run.smithi170.stdout:Architecture: amd64 2021-02-20T12:59:00.936 INFO:teuthology.orchestra.run.smithi170.stdout:Source: linux-meta 2021-02-20T12:59:00.936 INFO:teuthology.orchestra.run.smithi170.stdout:Version: 4.15.0.135.122 2021-02-20T12:59:00.937 INFO:teuthology.orchestra.run.smithi170.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-02-20T12:59:00.938 INFO:teuthology.orchestra.run.smithi170.stdout:Depends: linux-image-4.15.0-135-generic, linux-modules-extra-4.15.0-135-generic, linux-firmware, intel-microcode, amd64-microcode 2021-02-20T12:59:00.938 INFO:teuthology.orchestra.run.smithi170.stdout:Recommends: thermald 2021-02-20T12:59:00.939 INFO:teuthology.orchestra.run.smithi170.stdout:Description: Generic Linux kernel image 2021-02-20T12:59:00.940 INFO:teuthology.orchestra.run.smithi170.stdout: This package will always depend on the latest generic kernel image 2021-02-20T12:59:00.940 INFO:teuthology.orchestra.run.smithi170.stdout: available. 2021-02-20T12:59:00.941 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-134-generic Expected: 4.15.0-135-generic 2021-02-20T12:59:00.941 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2021-02-20T12:59:00.948 INFO:teuthology.orchestra.run.smithi183.stdout:4.15.0-134-generic 2021-02-20T12:59:00.948 INFO:teuthology.task.kernel:Running kernel on smithi183: 4.15.0-134-generic 2021-02-20T12:59:00.949 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get clean 2021-02-20T12:59:01.064 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get update 2021-02-20T12:59:01.333 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-02-20T12:59:01.334 INFO:teuthology.orchestra.run.smithi183.stdout:Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2021-02-20T12:59:01.392 INFO:teuthology.orchestra.run.smithi183.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-02-20T12:59:01.644 INFO:teuthology.orchestra.run.smithi183.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-02-20T12:59:01.734 INFO:teuthology.orchestra.run.smithi183.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,544 kB] 2021-02-20T12:59:01.866 INFO:teuthology.orchestra.run.smithi183.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,220 kB] 2021-02-20T12:59:02.171 INFO:teuthology.orchestra.run.smithi183.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [916 kB] 2021-02-20T12:59:02.189 INFO:teuthology.orchestra.run.smithi183.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [298 kB] 2021-02-20T12:59:02.194 INFO:teuthology.orchestra.run.smithi183.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [226 kB] 2021-02-20T12:59:02.198 INFO:teuthology.orchestra.run.smithi183.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [29.9 kB] 2021-02-20T12:59:02.199 INFO:teuthology.orchestra.run.smithi183.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [975 kB] 2021-02-20T12:59:02.209 INFO:teuthology.orchestra.run.smithi183.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,885 kB] 2021-02-20T12:59:02.267 INFO:teuthology.orchestra.run.smithi183.stdout:Get:13 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,109 kB] 2021-02-20T12:59:02.293 INFO:teuthology.orchestra.run.smithi183.stdout:Get:14 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [248 kB] 2021-02-20T12:59:02.312 INFO:teuthology.orchestra.run.smithi183.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [390 kB] 2021-02-20T12:59:02.329 INFO:teuthology.orchestra.run.smithi183.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [247 kB] 2021-02-20T12:59:02.340 INFO:teuthology.orchestra.run.smithi183.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [33.4 kB] 2021-02-20T12:59:02.341 INFO:teuthology.orchestra.run.smithi183.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,718 kB] 2021-02-20T12:59:02.383 INFO:teuthology.orchestra.run.smithi183.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,560 kB] 2021-02-20T12:59:02.433 INFO:teuthology.orchestra.run.smithi183.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [363 kB] 2021-02-20T12:59:02.439 INFO:teuthology.orchestra.run.smithi183.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [14.3 kB] 2021-02-20T12:59:02.440 INFO:teuthology.orchestra.run.smithi183.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [31.8 kB] 2021-02-20T12:59:02.440 INFO:teuthology.orchestra.run.smithi183.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [7,300 B] 2021-02-20T12:59:03.907 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 13.1 MB in 3s (5,036 kB/s) 2021-02-20T12:59:05.434 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2021-02-20T12:59:05.483 DEBUG:teuthology.orchestra.run.smithi183:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-02-20T12:59:05.561 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2021-02-20T12:59:05.804 INFO:teuthology.orchestra.run.smithi183.stdout:Building dependency tree... 2021-02-20T12:59:05.804 INFO:teuthology.orchestra.run.smithi183.stdout:Reading state information... 2021-02-20T12:59:05.885 INFO:teuthology.orchestra.run.smithi183.stderr:E: Unable to locate package linux-image-current-generic 2021-02-20T12:59:05.886 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-02-20T12:59:05.887 DEBUG:teuthology.orchestra.run.smithi183:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-02-20T12:59:05.964 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2021-02-20T12:59:06.211 INFO:teuthology.orchestra.run.smithi183.stdout:Building dependency tree... 2021-02-20T12:59:06.212 INFO:teuthology.orchestra.run.smithi183.stdout:Reading state information... 2021-02-20T12:59:06.385 INFO:teuthology.orchestra.run.smithi183.stdout:The following additional packages will be installed: 2021-02-20T12:59:06.386 INFO:teuthology.orchestra.run.smithi183.stdout: linux-generic linux-headers-4.15.0-135 linux-headers-4.15.0-135-generic 2021-02-20T12:59:06.386 INFO:teuthology.orchestra.run.smithi183.stdout: linux-headers-generic linux-image-4.15.0-135-generic 2021-02-20T12:59:06.386 INFO:teuthology.orchestra.run.smithi183.stdout: linux-modules-4.15.0-135-generic linux-modules-extra-4.15.0-135-generic 2021-02-20T12:59:06.389 INFO:teuthology.orchestra.run.smithi183.stdout:Suggested packages: 2021-02-20T12:59:06.390 INFO:teuthology.orchestra.run.smithi183.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2021-02-20T12:59:06.390 INFO:teuthology.orchestra.run.smithi183.stdout:Recommended packages: 2021-02-20T12:59:06.390 INFO:teuthology.orchestra.run.smithi183.stdout: thermald 2021-02-20T12:59:06.423 INFO:teuthology.orchestra.run.smithi183.stdout:The following NEW packages will be installed: 2021-02-20T12:59:06.423 INFO:teuthology.orchestra.run.smithi183.stdout: linux-headers-4.15.0-135 linux-headers-4.15.0-135-generic 2021-02-20T12:59:06.423 INFO:teuthology.orchestra.run.smithi183.stdout: linux-image-4.15.0-135-generic linux-modules-4.15.0-135-generic 2021-02-20T12:59:06.424 INFO:teuthology.orchestra.run.smithi183.stdout: linux-modules-extra-4.15.0-135-generic 2021-02-20T12:59:06.425 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages will be upgraded: 2021-02-20T12:59:06.426 INFO:teuthology.orchestra.run.smithi183.stdout: linux-generic linux-headers-generic linux-image-generic 2021-02-20T12:59:06.600 INFO:teuthology.orchestra.run.smithi183.stdout:3 upgraded, 5 newly installed, 0 to remove and 38 not upgraded. 2021-02-20T12:59:06.600 INFO:teuthology.orchestra.run.smithi183.stdout:Need to get 66.0 MB of archives. 2021-02-20T12:59:06.600 INFO:teuthology.orchestra.run.smithi183.stdout:After this operation, 334 MB of additional disk space will be used. 2021-02-20T12:59:06.601 INFO:teuthology.orchestra.run.smithi183.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-135-generic amd64 4.15.0-135.139 [13.1 MB] 2021-02-20T12:59:07.662 INFO:teuthology.orchestra.run.smithi183.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-135-generic amd64 4.15.0-135.139 [8,019 kB] 2021-02-20T12:59:08.494 INFO:teuthology.orchestra.run.smithi183.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-135-generic amd64 4.15.0-135.139 [32.8 MB] 2021-02-20T12:59:09.531 INFO:teuthology.orchestra.run.smithi183.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.135.122 [1,868 B] 2021-02-20T12:59:09.620 INFO:teuthology.orchestra.run.smithi183.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.135.122 [2,592 B] 2021-02-20T12:59:09.699 INFO:teuthology.orchestra.run.smithi183.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-135 all 4.15.0-135.139 [10.9 MB] 2021-02-20T12:59:10.101 INFO:teuthology.orchestra.run.smithi183.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-135-generic amd64 4.15.0-135.139 [1,097 kB] 2021-02-20T12:59:10.202 INFO:teuthology.orchestra.run.smithi183.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.135.122 [2,488 B] 2021-02-20T12:59:10.665 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 66.0 MB in 4s (17.5 MB/s) 2021-02-20T12:59:10.835 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-modules-4.15.0-135-generic. 2021-02-20T12:59:10.873 INFO:teuthology.orchestra.run.smithi183.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102079 files and directories currently installed.) 2021-02-20T12:59:10.876 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../0-linux-modules-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T12:59:10.923 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-modules-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T12:59:12.739 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-image-4.15.0-135-generic. 2021-02-20T12:59:12.753 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../1-linux-image-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T12:59:12.896 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-image-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T12:59:13.458 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-135-generic. 2021-02-20T12:59:13.471 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T12:59:13.508 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-modules-extra-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T12:59:17.255 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../3-linux-generic_4.15.0.135.122_amd64.deb ... 2021-02-20T12:59:17.408 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-generic (4.15.0.135.122) over (4.15.0.134.121) ... 2021-02-20T12:59:17.791 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.135.122_amd64.deb ... 2021-02-20T12:59:17.911 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-image-generic (4.15.0.135.122) over (4.15.0.134.121) ... 2021-02-20T12:59:18.290 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-headers-4.15.0-135. 2021-02-20T12:59:18.301 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../5-linux-headers-4.15.0-135_4.15.0-135.139_all.deb ... 2021-02-20T12:59:18.363 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-headers-4.15.0-135 (4.15.0-135.139) ... 2021-02-20T12:59:23.334 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-headers-4.15.0-135-generic. 2021-02-20T12:59:23.355 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../6-linux-headers-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T12:59:23.392 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-headers-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T12:59:25.970 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.135.122_amd64.deb ... 2021-02-20T12:59:26.094 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-headers-generic (4.15.0.135.122) over (4.15.0.134.121) ... 2021-02-20T12:59:26.470 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-modules-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T12:59:27.051 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-headers-4.15.0-135 (4.15.0-135.139) ... 2021-02-20T12:59:27.184 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-image-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T12:59:27.637 INFO:teuthology.orchestra.run.smithi183.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-135-generic 2021-02-20T12:59:27.638 INFO:teuthology.orchestra.run.smithi183.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-135-generic 2021-02-20T12:59:27.777 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-modules-extra-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T12:59:28.237 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-headers-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T12:59:28.388 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-image-generic (4.15.0.135.122) ... 2021-02-20T12:59:28.547 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-headers-generic (4.15.0.135.122) ... 2021-02-20T12:59:28.677 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-generic (4.15.0.135.122) ... 2021-02-20T12:59:28.824 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for linux-image-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T12:59:29.017 INFO:teuthology.orchestra.run.smithi183.stdout:/etc/kernel/postinst.d/initramfs-tools: 2021-02-20T12:59:29.017 INFO:teuthology.orchestra.run.smithi183.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-135-generic 2021-02-20T12:59:43.581 INFO:teuthology.orchestra.run.smithi183.stdout:/etc/kernel/postinst.d/zz-update-grub: 2021-02-20T12:59:43.582 INFO:teuthology.orchestra.run.smithi183.stdout:Sourcing file `/etc/default/grub' 2021-02-20T12:59:43.616 INFO:teuthology.orchestra.run.smithi183.stdout:Generating grub configuration file ... 2021-02-20T12:59:44.074 INFO:teuthology.orchestra.run.smithi183.stdout:Found linux image: /boot/vmlinuz-4.15.0-135-generic 2021-02-20T12:59:44.113 INFO:teuthology.orchestra.run.smithi183.stdout:Found initrd image: /boot/initrd.img-4.15.0-135-generic 2021-02-20T12:59:44.578 INFO:teuthology.orchestra.run.smithi183.stdout:Found linux image: /boot/vmlinuz-4.15.0-134-generic 2021-02-20T12:59:44.605 INFO:teuthology.orchestra.run.smithi183.stdout:Found initrd image: /boot/initrd.img-4.15.0-134-generic 2021-02-20T12:59:45.611 INFO:teuthology.orchestra.run.smithi183.stdout:done 2021-02-20T12:59:46.071 DEBUG:teuthology.orchestra.run.smithi183:> dpkg -s linux-image-generic 2021-02-20T12:59:46.093 INFO:teuthology.orchestra.run.smithi183.stdout:Package: linux-image-generic 2021-02-20T12:59:46.094 INFO:teuthology.orchestra.run.smithi183.stdout:Status: install ok installed 2021-02-20T12:59:46.094 INFO:teuthology.orchestra.run.smithi183.stdout:Priority: optional 2021-02-20T12:59:46.095 INFO:teuthology.orchestra.run.smithi183.stdout:Section: kernel 2021-02-20T12:59:46.095 INFO:teuthology.orchestra.run.smithi183.stdout:Installed-Size: 17 2021-02-20T12:59:46.095 INFO:teuthology.orchestra.run.smithi183.stdout:Maintainer: Ubuntu Kernel Team 2021-02-20T12:59:46.095 INFO:teuthology.orchestra.run.smithi183.stdout:Architecture: amd64 2021-02-20T12:59:46.096 INFO:teuthology.orchestra.run.smithi183.stdout:Source: linux-meta 2021-02-20T12:59:46.096 INFO:teuthology.orchestra.run.smithi183.stdout:Version: 4.15.0.135.122 2021-02-20T12:59:46.096 INFO:teuthology.orchestra.run.smithi183.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-02-20T12:59:46.096 INFO:teuthology.orchestra.run.smithi183.stdout:Depends: linux-image-4.15.0-135-generic, linux-modules-extra-4.15.0-135-generic, linux-firmware, intel-microcode, amd64-microcode 2021-02-20T12:59:46.097 INFO:teuthology.orchestra.run.smithi183.stdout:Recommends: thermald 2021-02-20T12:59:46.097 INFO:teuthology.orchestra.run.smithi183.stdout:Description: Generic Linux kernel image 2021-02-20T12:59:46.098 INFO:teuthology.orchestra.run.smithi183.stdout: This package will always depend on the latest generic kernel image 2021-02-20T12:59:46.098 INFO:teuthology.orchestra.run.smithi183.stdout: available. 2021-02-20T12:59:46.099 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-134-generic Expected: 4.15.0-135-generic 2021-02-20T12:59:46.099 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2021-02-20T12:59:46.099 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-02-20T12:59:46.100 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-02-20T12:59:46.101 INFO:teuthology.task.kernel:Installing distro kernel on cluster1.mon.a... 2021-02-20T12:59:46.102 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi170.front.sepia.ceph.com, path=None, version=distro) 2021-02-20T12:59:46.102 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get clean 2021-02-20T12:59:46.224 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get update 2021-02-20T12:59:46.343 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-02-20T12:59:46.459 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-02-20T12:59:46.540 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-02-20T12:59:46.621 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-02-20T12:59:48.210 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2021-02-20T12:59:48.258 DEBUG:teuthology.orchestra.run.smithi170:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-02-20T12:59:48.342 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2021-02-20T12:59:48.588 INFO:teuthology.orchestra.run.smithi170.stdout:Building dependency tree... 2021-02-20T12:59:48.589 INFO:teuthology.orchestra.run.smithi170.stdout:Reading state information... 2021-02-20T12:59:48.671 INFO:teuthology.orchestra.run.smithi170.stderr:E: Unable to locate package linux-image-current-generic 2021-02-20T12:59:48.672 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-02-20T12:59:48.673 DEBUG:teuthology.orchestra.run.smithi170:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-02-20T12:59:48.752 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2021-02-20T12:59:48.983 INFO:teuthology.orchestra.run.smithi170.stdout:Building dependency tree... 2021-02-20T12:59:48.984 INFO:teuthology.orchestra.run.smithi170.stdout:Reading state information... 2021-02-20T12:59:49.188 INFO:teuthology.orchestra.run.smithi170.stdout:linux-image-generic is already the newest version (4.15.0.135.122). 2021-02-20T12:59:49.188 INFO:teuthology.orchestra.run.smithi170.stdout:0 upgraded, 0 newly installed, 0 to remove and 38 not upgraded. 2021-02-20T12:59:49.189 DEBUG:teuthology.orchestra.run.smithi170:> dpkg -s linux-image-generic 2021-02-20T12:59:49.204 INFO:teuthology.orchestra.run.smithi170.stdout:Package: linux-image-generic 2021-02-20T12:59:49.204 INFO:teuthology.orchestra.run.smithi170.stdout:Status: install ok installed 2021-02-20T12:59:49.204 INFO:teuthology.orchestra.run.smithi170.stdout:Priority: optional 2021-02-20T12:59:49.205 INFO:teuthology.orchestra.run.smithi170.stdout:Section: kernel 2021-02-20T12:59:49.205 INFO:teuthology.orchestra.run.smithi170.stdout:Installed-Size: 17 2021-02-20T12:59:49.205 INFO:teuthology.orchestra.run.smithi170.stdout:Maintainer: Ubuntu Kernel Team 2021-02-20T12:59:49.205 INFO:teuthology.orchestra.run.smithi170.stdout:Architecture: amd64 2021-02-20T12:59:49.205 INFO:teuthology.orchestra.run.smithi170.stdout:Source: linux-meta 2021-02-20T12:59:49.206 INFO:teuthology.orchestra.run.smithi170.stdout:Version: 4.15.0.135.122 2021-02-20T12:59:49.206 INFO:teuthology.orchestra.run.smithi170.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-02-20T12:59:49.206 INFO:teuthology.orchestra.run.smithi170.stdout:Depends: linux-image-4.15.0-135-generic, linux-modules-extra-4.15.0-135-generic, linux-firmware, intel-microcode, amd64-microcode 2021-02-20T12:59:49.206 INFO:teuthology.orchestra.run.smithi170.stdout:Recommends: thermald 2021-02-20T12:59:49.206 INFO:teuthology.orchestra.run.smithi170.stdout:Description: Generic Linux kernel image 2021-02-20T12:59:49.207 INFO:teuthology.orchestra.run.smithi170.stdout: This package will always depend on the latest generic kernel image 2021-02-20T12:59:49.207 INFO:teuthology.orchestra.run.smithi170.stdout: available. 2021-02-20T12:59:49.207 DEBUG:teuthology.orchestra.run.smithi170:> mktemp 2021-02-20T12:59:49.250 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.ckYH95MUvI 2021-02-20T12:59:49.250 DEBUG:teuthology.orchestra.run.smithi170:> sudo cp /boot/grub/grub.cfg /tmp/tmp.ckYH95MUvI 2021-02-20T12:59:49.310 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.ckYH95MUvI 2021-02-20T12:59:49.434 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.ckYH95MUvI is 8KB 2021-02-20T12:59:49.493 DEBUG:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.ckYH95MUvI 2021-02-20T12:59:49.536 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2021-02-20T12:59:49.591 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T12:59:49.595 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/grub.d/01_ceph_kernel 2021-02-20T12:59:49.596 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2021-02-20T12:59:49.702 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-135-generic 2021-02-20T12:59:49.705 DEBUG:teuthology.orchestra.run.smithi170:> sudo update-grub 2021-02-20T12:59:50.985 INFO:teuthology.orchestra.run.smithi170.stderr:Sourcing file `/etc/default/grub' 2021-02-20T12:59:51.006 INFO:teuthology.orchestra.run.smithi170.stderr:Generating grub configuration file ... 2021-02-20T12:59:51.325 INFO:teuthology.orchestra.run.smithi170.stderr:Found linux image: /boot/vmlinuz-4.15.0-135-generic 2021-02-20T12:59:51.336 INFO:teuthology.orchestra.run.smithi170.stderr:Found initrd image: /boot/initrd.img-4.15.0-135-generic 2021-02-20T12:59:51.685 INFO:teuthology.orchestra.run.smithi170.stderr:Found linux image: /boot/vmlinuz-4.15.0-134-generic 2021-02-20T12:59:51.699 INFO:teuthology.orchestra.run.smithi170.stderr:Found initrd image: /boot/initrd.img-4.15.0-134-generic 2021-02-20T12:59:52.321 INFO:teuthology.orchestra.run.smithi170.stderr:done 2021-02-20T12:59:52.322 DEBUG:teuthology.orchestra.run.smithi170:> sudo shutdown -r now 2021-02-20T12:59:52.325 INFO:teuthology.task.kernel:Installing distro kernel on cluster2.mon.a... 2021-02-20T12:59:52.325 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi183.front.sepia.ceph.com, path=None, version=distro) 2021-02-20T12:59:52.326 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get clean 2021-02-20T12:59:52.367 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get update 2021-02-20T12:59:52.656 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-02-20T12:59:52.656 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-02-20T12:59:52.724 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-02-20T12:59:52.804 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-02-20T12:59:54.538 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2021-02-20T12:59:54.586 DEBUG:teuthology.orchestra.run.smithi183:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-02-20T12:59:54.662 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2021-02-20T12:59:54.897 INFO:teuthology.orchestra.run.smithi183.stdout:Building dependency tree... 2021-02-20T12:59:54.897 INFO:teuthology.orchestra.run.smithi183.stdout:Reading state information... 2021-02-20T12:59:54.979 INFO:teuthology.orchestra.run.smithi183.stderr:E: Unable to locate package linux-image-current-generic 2021-02-20T12:59:54.980 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-02-20T12:59:54.981 DEBUG:teuthology.orchestra.run.smithi183:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-02-20T12:59:55.053 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2021-02-20T12:59:55.304 INFO:teuthology.orchestra.run.smithi183.stdout:Building dependency tree... 2021-02-20T12:59:55.305 INFO:teuthology.orchestra.run.smithi183.stdout:Reading state information... 2021-02-20T12:59:55.504 INFO:teuthology.orchestra.run.smithi183.stdout:linux-image-generic is already the newest version (4.15.0.135.122). 2021-02-20T12:59:55.505 INFO:teuthology.orchestra.run.smithi183.stdout:0 upgraded, 0 newly installed, 0 to remove and 38 not upgraded. 2021-02-20T12:59:55.506 DEBUG:teuthology.orchestra.run.smithi183:> dpkg -s linux-image-generic 2021-02-20T12:59:55.570 INFO:teuthology.orchestra.run.smithi183.stdout:Package: linux-image-generic 2021-02-20T12:59:55.570 INFO:teuthology.orchestra.run.smithi183.stdout:Status: install ok installed 2021-02-20T12:59:55.570 INFO:teuthology.orchestra.run.smithi183.stdout:Priority: optional 2021-02-20T12:59:55.570 INFO:teuthology.orchestra.run.smithi183.stdout:Section: kernel 2021-02-20T12:59:55.571 INFO:teuthology.orchestra.run.smithi183.stdout:Installed-Size: 17 2021-02-20T12:59:55.571 INFO:teuthology.orchestra.run.smithi183.stdout:Maintainer: Ubuntu Kernel Team 2021-02-20T12:59:55.571 INFO:teuthology.orchestra.run.smithi183.stdout:Architecture: amd64 2021-02-20T12:59:55.571 INFO:teuthology.orchestra.run.smithi183.stdout:Source: linux-meta 2021-02-20T12:59:55.572 INFO:teuthology.orchestra.run.smithi183.stdout:Version: 4.15.0.135.122 2021-02-20T12:59:55.572 INFO:teuthology.orchestra.run.smithi183.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-02-20T12:59:55.572 INFO:teuthology.orchestra.run.smithi183.stdout:Depends: linux-image-4.15.0-135-generic, linux-modules-extra-4.15.0-135-generic, linux-firmware, intel-microcode, amd64-microcode 2021-02-20T12:59:55.572 INFO:teuthology.orchestra.run.smithi183.stdout:Recommends: thermald 2021-02-20T12:59:55.572 INFO:teuthology.orchestra.run.smithi183.stdout:Description: Generic Linux kernel image 2021-02-20T12:59:55.573 INFO:teuthology.orchestra.run.smithi183.stdout: This package will always depend on the latest generic kernel image 2021-02-20T12:59:55.573 INFO:teuthology.orchestra.run.smithi183.stdout: available. 2021-02-20T12:59:55.573 DEBUG:teuthology.orchestra.run.smithi183:> mktemp 2021-02-20T12:59:55.618 INFO:teuthology.orchestra.run.smithi183.stdout:/tmp/tmp.v5CQnmEoqa 2021-02-20T12:59:55.624 DEBUG:teuthology.orchestra.run.smithi183:> sudo cp /boot/grub/grub.cfg /tmp/tmp.v5CQnmEoqa 2021-02-20T12:59:55.675 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0666 /tmp/tmp.v5CQnmEoqa 2021-02-20T12:59:55.804 DEBUG:teuthology.orchestra.remote:smithi183:/tmp/tmp.v5CQnmEoqa is 8KB 2021-02-20T12:59:55.815 DEBUG:teuthology.orchestra.run.smithi183:> rm -fr /tmp/tmp.v5CQnmEoqa 2021-02-20T12:59:55.821 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2021-02-20T12:59:55.879 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T12:59:55.880 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/grub.d/01_ceph_kernel 2021-02-20T12:59:55.881 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2021-02-20T12:59:55.962 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-135-generic 2021-02-20T12:59:55.962 DEBUG:teuthology.orchestra.run.smithi183:> sudo update-grub 2021-02-20T12:59:57.542 INFO:teuthology.orchestra.run.smithi183.stderr:Sourcing file `/etc/default/grub' 2021-02-20T12:59:57.563 INFO:teuthology.orchestra.run.smithi183.stderr:Generating grub configuration file ... 2021-02-20T12:59:57.895 INFO:teuthology.orchestra.run.smithi183.stderr:Found linux image: /boot/vmlinuz-4.15.0-135-generic 2021-02-20T12:59:57.907 INFO:teuthology.orchestra.run.smithi183.stderr:Found initrd image: /boot/initrd.img-4.15.0-135-generic 2021-02-20T12:59:58.305 INFO:teuthology.orchestra.run.smithi183.stderr:Found linux image: /boot/vmlinuz-4.15.0-134-generic 2021-02-20T12:59:58.318 INFO:teuthology.orchestra.run.smithi183.stderr:Found initrd image: /boot/initrd.img-4.15.0-134-generic 2021-02-20T12:59:58.939 INFO:teuthology.orchestra.run.smithi183.stderr:done 2021-02-20T12:59:58.941 DEBUG:teuthology.orchestra.run.smithi183:> sudo shutdown -r now 2021-02-20T13:00:28.970 INFO:teuthology.misc:Re-opening connections... 2021-02-20T13:00:28.971 INFO:teuthology.misc:trying to connect to ubuntu@smithi170.front.sepia.ceph.com 2021-02-20T13:00:28.972 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-20T13:00:28.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T13:00:47.426 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2021-02-20T13:00:47.431 INFO:teuthology.misc:trying to connect to ubuntu@smithi183.front.sepia.ceph.com 2021-02-20T13:00:47.432 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-20T13:00:47.434 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T13:01:47.436 DEBUG:teuthology.orchestra.remote:timed out 2021-02-20T13:01:47.457 DEBUG:teuthology.misc:waited 78.48541975021362 2021-02-20T13:01:48.457 INFO:teuthology.misc:trying to connect to ubuntu@smithi170.front.sepia.ceph.com 2021-02-20T13:01:48.458 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-20T13:01:48.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T13:01:48.605 DEBUG:teuthology.orchestra.run.smithi170:> true 2021-02-20T13:01:49.067 DEBUG:teuthology.misc:waited 80.09596419334412 2021-02-20T13:01:50.068 INFO:teuthology.misc:trying to connect to ubuntu@smithi183.front.sepia.ceph.com 2021-02-20T13:01:50.080 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-20T13:01:50.081 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T13:01:50.356 DEBUG:teuthology.orchestra.run.smithi183:> true 2021-02-20T13:01:50.864 DEBUG:teuthology.misc:waited 81.89312720298767 2021-02-20T13:01:51.865 INFO:teuthology.task.kernel:Checking client cluster1.mon.a for new kernel version... 2021-02-20T13:01:51.866 INFO:teuthology.task.kernel:Checking kernel version of cluster1.mon.a, want "4.15.0-135-generic"... 2021-02-20T13:01:51.867 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2021-02-20T13:01:51.874 INFO:teuthology.orchestra.run.smithi170.stdout:4.15.0-135-generic 2021-02-20T13:01:51.875 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-135-generic vs 4.15.0-135-generic 2021-02-20T13:01:51.875 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-02-20T13:01:51.876 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2021-02-20T13:01:51.876 INFO:teuthology.task.kernel:Checking client cluster2.mon.a for new kernel version... 2021-02-20T13:01:51.876 INFO:teuthology.task.kernel:Checking kernel version of cluster2.mon.a, want "4.15.0-135-generic"... 2021-02-20T13:01:51.876 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2021-02-20T13:01:51.883 INFO:teuthology.orchestra.run.smithi183.stdout:4.15.0-135-generic 2021-02-20T13:01:51.883 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-135-generic vs 4.15.0-135-generic 2021-02-20T13:01:51.884 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-02-20T13:01:51.884 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2021-02-20T13:01:52.886 INFO:teuthology.task.kernel:Enabling kdb on cluster1.mon.a... 2021-02-20T13:01:52.887 DEBUG:teuthology.orchestra.run.smithi170:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-02-20T13:01:52.919 INFO:teuthology.orchestra.run.smithi170.stdout:ttyS1 2021-02-20T13:01:52.945 INFO:teuthology.task.kernel:Enabling kdb on cluster2.mon.a... 2021-02-20T13:01:52.946 DEBUG:teuthology.orchestra.run.smithi183:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-02-20T13:01:52.992 INFO:teuthology.orchestra.run.smithi183.stdout:ttyS1 2021-02-20T13:01:53.017 INFO:teuthology.run_tasks:Running task internal.base... 2021-02-20T13:01:53.030 INFO:teuthology.task.internal:Creating test directory... 2021-02-20T13:01:53.032 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-02-20T13:01:53.045 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-02-20T13:01:53.059 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2021-02-20T13:01:53.088 INFO:teuthology.run_tasks:Running task internal.archive... 2021-02-20T13:01:53.118 INFO:teuthology.task.internal:Creating archive directory... 2021-02-20T13:01:53.119 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-02-20T13:01:53.161 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-02-20T13:01:53.226 INFO:teuthology.run_tasks:Running task internal.coredump... 2021-02-20T13:01:53.247 INFO:teuthology.task.internal:Enabling coredump saving... 2021-02-20T13:01:53.248 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-02-20T13:01:53.264 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-02-20T13:01:53.280 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-02-20T13:01:53.283 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-02-20T13:01:53.287 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-02-20T13:01:53.293 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-02-20T13:01:53.294 INFO:teuthology.run_tasks:Running task internal.sudo... 2021-02-20T13:01:53.308 INFO:teuthology.task.internal:Configuring sudo... 2021-02-20T13:01:53.308 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-02-20T13:01:53.332 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-02-20T13:01:53.354 INFO:teuthology.run_tasks:Running task internal.syslog... 2021-02-20T13:01:53.385 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2021-02-20T13:01:53.393 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-02-20T13:01:53.402 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-02-20T13:01:53.415 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-02-20T13:01:53.466 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-02-20T13:01:53.513 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:01:53.514 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-02-20T13:01:53.578 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-02-20T13:01:53.586 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-02-20T13:01:53.632 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:01:53.633 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-02-20T13:01:53.698 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2021-02-20T13:01:53.702 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2021-02-20T13:01:53.785 INFO:teuthology.run_tasks:Running task internal.timer... 2021-02-20T13:01:53.806 INFO:teuthology.task.internal:Starting timer... 2021-02-20T13:01:53.807 INFO:teuthology.run_tasks:Running task pcp... 2021-02-20T13:01:53.852 INFO:teuthology.run_tasks:Running task selinux... 2021-02-20T13:01:53.895 DEBUG:teuthology.task.selinux:Excluding smithi170: OS 'ubuntu' does not support SELinux 2021-02-20T13:01:53.896 DEBUG:teuthology.task.selinux:Excluding smithi183: OS 'ubuntu' does not support SELinux 2021-02-20T13:01:53.896 DEBUG:teuthology.task.selinux:Getting current SELinux state 2021-02-20T13:01:53.897 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2021-02-20T13:01:53.897 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2021-02-20T13:01:53.912 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2021-02-20T13:01:53.925 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated or references a specific commit; assuming it is current 2021-02-20T13:01:53.928 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to origin/master 2021-02-20T13:01:54.195 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2021-02-20T13:01:54.201 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi170.front.sepia.ceph.com,smithi183.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2021-02-20T13:01:55.454 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2021-02-20T13:01:58.389 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2021-02-20T13:02:02.526 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2021-02-20T13:02:02.579 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:02 +0000 (0:00:00.228) 0:00:00.228 ***** 2021-02-20T13:02:02.601 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:02 +0000 (0:00:00.019) 0:00:00.248 ***** 2021-02-20T13:02:05.051 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2021-02-20T13:02:05.056 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] 2021-02-20T13:02:05.190 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:05 +0000 (0:00:02.590) 0:00:02.839 ***** 2021-02-20T13:02:05.222 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] 2021-02-20T13:02:05.486 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:05 +0000 (0:00:00.290) 0:00:03.129 ***** 2021-02-20T13:02:05.707 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2021-02-20T13:02:05.709 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-02-20T13:02:05.841 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:05 +0000 (0:00:00.359) 0:00:03.489 ***** 2021-02-20T13:02:05.853 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-02-20T13:02:05.903 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:05 +0000 (0:00:00.063) 0:00:03.552 ***** 2021-02-20T13:02:06.604 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2021-02-20T13:02:06.606 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-02-20T13:02:06.667 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:06 +0000 (0:00:00.763) 0:00:04.316 ***** 2021-02-20T13:02:06.675 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-02-20T13:02:06.714 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:06 +0000 (0:00:00.047) 0:00:04.363 ***** 2021-02-20T13:02:07.188 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2021-02-20T13:02:07.191 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.065793", "end": "2021-02-20 13:02:06.885906", "rc": 0, "start": "2021-02-20 13:02:06.820113", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2021-02-20T13:02:07.259 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:07 +0000 (0:00:00.545) 0:00:04.909 ***** 2021-02-20T13:02:07.283 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.073208", "end": "2021-02-20 13:02:07.495912", "rc": 0, "start": "2021-02-20 13:02:07.422704", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2021-02-20T13:02:07.321 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:07 +0000 (0:00:00.061) 0:00:04.971 ***** 2021-02-20T13:02:07.581 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2021-02-20T13:02:07.584 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T13:02:07.645 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:07 +0000 (0:00:00.323) 0:00:05.294 ***** 2021-02-20T13:02:07.666 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T13:02:07.803 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:07 +0000 (0:00:00.157) 0:00:05.452 ***** 2021-02-20T13:02:08.831 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613826127.6756167-261733211013523/source", "state": "file", "uid": 0} 2021-02-20T13:02:08.832 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613826127.8507364-266243258232213/source", "state": "file", "uid": 0} 2021-02-20T13:02:08.895 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:08 +0000 (0:00:01.091) 0:00:06.544 ***** 2021-02-20T13:02:09.007 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:09 +0000 (0:00:00.111) 0:00:06.656 ***** 2021-02-20T13:02:09.472 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2021-02-20T13:02:09.475 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2021-02-20T13:02:09.477 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-02-20T13:02:09.585 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2021-02-20T13:02:09.587 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-02-20T13:02:09.766 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2021-02-20T13:02:09.785 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2021-02-20T13:02:10.143 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:10 +0000 (0:00:01.134) 0:00:07.790 ***** 2021-02-20T13:02:10.246 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:10 +0000 (0:00:00.105) 0:00:07.896 ***** 2021-02-20T13:02:11.004 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi170.front.sepia.ceph.com] 2021-02-20T13:02:11.110 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] 2021-02-20T13:02:11.255 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2021-02-20T13:02:11.257 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:11 +0000 (0:00:01.008) 0:00:08.904 ***** 2021-02-20T13:02:11.421 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:11 +0000 (0:00:00.165) 0:00:09.070 ***** 2021-02-20T13:02:12.289 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2021-02-20T13:02:12.292 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.483644", "end": "2021-02-20 13:02:12.237659", "rc": 0, "start": "2021-02-20 13:02:11.754015", "stderr": "", "stderr_lines": [], "stdout": "df8946f08e870e2d2565b79099289ca48bd825f3", "stdout_lines": ["df8946f08e870e2d2565b79099289ca48bd825f3"]} 2021-02-20T13:02:12.353 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:12 +0000 (0:00:00.932) 0:00:10.002 ***** 2021-02-20T13:02:12.372 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.489273", "end": "2021-02-20 13:02:12.236946", "rc": 0, "start": "2021-02-20 13:02:11.747673", "stderr": "", "stderr_lines": [], "stdout": "df8946f08e870e2d2565b79099289ca48bd825f3", "stdout_lines": ["df8946f08e870e2d2565b79099289ca48bd825f3"]} 2021-02-20T13:02:12.429 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:12 +0000 (0:00:00.075) 0:00:10.078 ***** 2021-02-20T13:02:12.558 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2021-02-20T13:02:12.559 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.011102", "end": "2021-02-20 13:02:12.264105", "failed_when_result": false, "rc": 0, "start": "2021-02-20 13:02:12.253003", "stderr": "", "stderr_lines": [], "stdout": "0caff10a478f2674721da1965013b225f3ea5800", "stdout_lines": ["0caff10a478f2674721da1965013b225f3ea5800"]} 2021-02-20T13:02:12.627 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:12 +0000 (0:00:00.198) 0:00:10.276 ***** 2021-02-20T13:02:12.682 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:02:12.684 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2021-02-20T13:02:12.685 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.020834", "end": "2021-02-20 13:02:12.971623", "failed_when_result": false, "rc": 0, "start": "2021-02-20 13:02:12.950789", "stderr": "", "stderr_lines": [], "stdout": "0caff10a478f2674721da1965013b225f3ea5800", "stdout_lines": ["0caff10a478f2674721da1965013b225f3ea5800"]} 2021-02-20T13:02:12.769 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:12 +0000 (0:00:00.142) 0:00:10.419 ***** 2021-02-20T13:02:12.826 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:12 +0000 (0:00:00.056) 0:00:10.475 ***** 2021-02-20T13:02:12.845 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2021-02-20T13:02:12.847 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:02:12.935 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:12 +0000 (0:00:00.109) 0:00:10.585 ***** 2021-02-20T13:02:12.947 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2021-02-20T13:02:12.949 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:02:13.031 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:13 +0000 (0:00:00.095) 0:00:10.680 ***** 2021-02-20T13:02:13.039 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2021-02-20T13:02:13.041 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:02:13.087 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:13 +0000 (0:00:00.055) 0:00:10.736 ***** 2021-02-20T13:02:13.108 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2021-02-20T13:02:13.110 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:02:13.151 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:13 +0000 (0:00:00.061) 0:00:10.798 ***** 2021-02-20T13:02:13.175 INFO:teuthology.task.ansible.out: 2021-02-20T13:02:13.179 INFO:teuthology.task.ansible.out:TASK [users : Filter the managed_users list] *********************************** 2021-02-20T13:02:13.181 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:02:13.226 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:13 +0000 (0:00:00.075) 0:00:10.874 ***** 2021-02-20T13:02:13.244 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2021-02-20T13:02:13.247 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:02:13.300 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:13 +0000 (0:00:00.074) 0:00:10.949 ***** 2021-02-20T13:02:13.307 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2021-02-20T13:02:13.309 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:02:13.351 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:13 +0000 (0:00:00.048) 0:00:11.000 ***** 2021-02-20T13:02:13.369 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2021-02-20T13:02:13.371 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:02:13.416 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:13 +0000 (0:00:00.065) 0:00:11.065 ***** 2021-02-20T13:02:13.454 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2021-02-20T13:02:13.456 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:02:13.499 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:13 +0000 (0:00:00.082) 0:00:11.148 ***** 2021-02-20T13:02:13.539 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2021-02-20T13:02:13.541 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T13:02:13.603 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:13 +0000 (0:00:00.104) 0:00:11.253 ***** 2021-02-20T13:02:13.706 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T13:02:13.746 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:13 +0000 (0:00:00.143) 0:00:11.396 ***** 2021-02-20T13:02:13.888 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2021-02-20T13:02:13.890 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-02-20T13:02:13.988 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-02-20T13:02:14.072 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-02-20T13:02:14.190 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-02-20T13:02:14.254 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-02-20T13:02:14.360 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-02-20T13:02:14.444 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-02-20T13:02:14.533 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-02-20T13:02:14.600 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-02-20T13:02:14.677 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-02-20T13:02:14.762 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-02-20T13:02:14.857 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-02-20T13:02:14.947 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-02-20T13:02:15.033 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-02-20T13:02:15.123 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-02-20T13:02:15.222 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-02-20T13:02:15.290 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-02-20T13:02:15.391 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-02-20T13:02:15.456 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-02-20T13:02:15.557 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-02-20T13:02:15.658 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-02-20T13:02:15.762 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-02-20T13:02:15.812 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-02-20T13:02:15.947 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-02-20T13:02:15.960 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-02-20T13:02:16.092 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-02-20T13:02:16.131 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-02-20T13:02:16.279 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-02-20T13:02:16.281 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-02-20T13:02:16.440 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-02-20T13:02:16.472 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-02-20T13:02:16.621 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-02-20T13:02:16.632 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-02-20T13:02:16.826 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-02-20T13:02:16.990 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-02-20T13:02:17.152 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-02-20T13:02:17.321 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-02-20T13:02:17.486 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-02-20T13:02:17.642 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-02-20T13:02:17.771 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-02-20T13:02:17.815 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-02-20T13:02:17.919 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-02-20T13:02:17.969 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-02-20T13:02:18.130 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-02-20T13:02:18.146 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-02-20T13:02:18.302 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-02-20T13:02:18.317 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-02-20T13:02:18.503 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-02-20T13:02:18.536 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-02-20T13:02:18.647 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-02-20T13:02:18.745 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-02-20T13:02:18.817 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-02-20T13:02:18.949 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-02-20T13:02:18.971 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-02-20T13:02:19.143 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-02-20T13:02:19.154 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-02-20T13:02:19.296 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-02-20T13:02:19.328 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-02-20T13:02:19.429 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-02-20T13:02:19.483 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-02-20T13:02:19.577 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-02-20T13:02:19.679 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-02-20T13:02:19.717 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-02-20T13:02:19.877 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-02-20T13:02:19.879 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-02-20T13:02:20.063 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-02-20T13:02:20.091 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-02-20T13:02:20.096 INFO:teuthology.task.ansible.out: 2021-02-20T13:02:20.248 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-02-20T13:02:20.270 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-02-20T13:02:20.431 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-02-20T13:02:20.443 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-02-20T13:02:20.571 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-02-20T13:02:20.614 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-02-20T13:02:20.760 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-02-20T13:02:20.824 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-02-20T13:02:20.952 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-02-20T13:02:20.986 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-02-20T13:02:21.126 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-02-20T13:02:21.200 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-02-20T13:02:21.289 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-02-20T13:02:21.351 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-02-20T13:02:21.426 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-02-20T13:02:21.503 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-02-20T13:02:21.588 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-02-20T13:02:21.673 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-02-20T13:02:21.748 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-02-20T13:02:21.834 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-02-20T13:02:21.942 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-02-20T13:02:22.015 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-02-20T13:02:22.133 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-02-20T13:02:22.283 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-02-20T13:02:22.402 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-02-20T13:02:22.458 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-02-20T13:02:22.603 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-02-20T13:02:22.674 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-02-20T13:02:22.771 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-02-20T13:02:22.862 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-02-20T13:02:22.962 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-02-20T13:02:23.039 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-02-20T13:02:23.158 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-02-20T13:02:23.229 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-02-20T13:02:23.295 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-02-20T13:02:23.383 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-02-20T13:02:23.453 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-02-20T13:02:23.540 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-02-20T13:02:23.606 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-02-20T13:02:23.729 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-02-20T13:02:23.787 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-02-20T13:02:23.896 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-02-20T13:02:23.973 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-02-20T13:02:24.080 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-02-20T13:02:24.193 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-02-20T13:02:24.366 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-02-20T13:02:24.379 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-02-20T13:02:24.565 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-02-20T13:02:24.567 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-02-20T13:02:24.750 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-02-20T13:02:24.763 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-02-20T13:02:24.915 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-02-20T13:02:24.928 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-02-20T13:02:25.098 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-02-20T13:02:25.121 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-02-20T13:02:25.260 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-02-20T13:02:25.303 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-02-20T13:02:25.446 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-02-20T13:02:25.491 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-02-20T13:02:25.620 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-02-20T13:02:25.664 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-02-20T13:02:25.795 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-02-20T13:02:25.808 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-02-20T13:02:25.974 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-02-20T13:02:25.985 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-02-20T13:02:26.182 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-02-20T13:02:26.184 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-02-20T13:02:26.345 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-02-20T13:02:26.357 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-02-20T13:02:26.513 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-02-20T13:02:26.559 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-02-20T13:02:26.714 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-02-20T13:02:26.716 INFO:teuthology.task.ansible.out: 2021-02-20T13:02:26.780 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-02-20T13:02:26.910 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-02-20T13:02:26.955 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-02-20T13:02:27.134 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-02-20T13:02:27.263 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-02-20T13:02:27.307 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-02-20T13:02:27.651 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-02-20T13:02:27.675 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-02-20T13:02:27.833 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-02-20T13:02:27.978 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-02-20T13:02:28.118 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-02-20T13:02:28.248 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-02-20T13:02:28.319 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-02-20T13:02:28.320 INFO:teuthology.task.ansible.out: 2021-02-20T13:02:28.517 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-02-20T13:02:28.519 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-02-20T13:02:28.708 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-02-20T13:02:28.744 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-02-20T13:02:28.944 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-02-20T13:02:28.963 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-02-20T13:02:29.140 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-02-20T13:02:29.155 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-02-20T13:02:29.341 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-02-20T13:02:29.344 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-02-20T13:02:29.485 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-02-20T13:02:29.540 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-02-20T13:02:29.675 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-02-20T13:02:29.688 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-02-20T13:02:29.868 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-02-20T13:02:29.869 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-02-20T13:02:30.031 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-02-20T13:02:30.063 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-02-20T13:02:30.207 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-02-20T13:02:30.229 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-02-20T13:02:30.356 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-02-20T13:02:30.367 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-02-20T13:02:30.512 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-02-20T13:02:30.524 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-02-20T13:02:30.687 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-02-20T13:02:30.699 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-02-20T13:02:30.869 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-02-20T13:02:30.902 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-02-20T13:02:31.030 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-02-20T13:02:31.073 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-02-20T13:02:31.180 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-02-20T13:02:31.223 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-02-20T13:02:31.355 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-02-20T13:02:31.399 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-02-20T13:02:31.528 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-02-20T13:02:31.572 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-02-20T13:02:31.722 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-02-20T13:02:31.759 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-02-20T13:02:31.898 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-02-20T13:02:31.946 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-02-20T13:02:32.087 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-02-20T13:02:32.100 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-02-20T13:02:32.267 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-02-20T13:02:32.283 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-02-20T13:02:32.472 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-02-20T13:02:32.487 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-02-20T13:02:32.658 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-02-20T13:02:32.703 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-02-20T13:02:32.833 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-02-20T13:02:32.955 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-02-20T13:02:32.999 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-02-20T13:02:33.113 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-02-20T13:02:33.157 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-02-20T13:02:33.276 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-02-20T13:02:33.362 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-02-20T13:02:33.445 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-02-20T13:02:33.597 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-02-20T13:02:33.630 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-02-20T13:02:33.797 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-02-20T13:02:33.831 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-02-20T13:02:33.963 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-02-20T13:02:34.001 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-02-20T13:02:34.134 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-02-20T13:02:34.178 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-02-20T13:02:34.313 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-02-20T13:02:34.347 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-02-20T13:02:34.467 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-02-20T13:02:34.507 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-02-20T13:02:34.671 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-02-20T13:02:34.683 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-02-20T13:02:34.851 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-02-20T13:02:34.911 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-02-20T13:02:35.034 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-02-20T13:02:35.067 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-02-20T13:02:35.184 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-02-20T13:02:35.295 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-02-20T13:02:35.361 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-02-20T13:02:35.477 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-02-20T13:02:35.479 INFO:teuthology.task.ansible.out: 2021-02-20T13:02:35.514 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-02-20T13:02:35.737 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-02-20T13:02:35.739 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-02-20T13:02:35.934 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-02-20T13:02:35.958 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-02-20T13:02:36.100 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-02-20T13:02:36.177 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-02-20T13:02:36.267 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-02-20T13:02:36.367 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-02-20T13:02:36.478 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-02-20T13:02:36.575 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-02-20T13:02:36.686 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-02-20T13:02:36.784 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-02-20T13:02:36.848 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-02-20T13:02:36.996 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-02-20T13:02:37.050 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-02-20T13:02:37.177 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-02-20T13:02:37.221 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-02-20T13:02:37.407 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-02-20T13:02:37.409 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-02-20T13:02:37.548 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-02-20T13:02:37.574 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-02-20T13:02:37.725 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-02-20T13:02:37.764 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-02-20T13:02:37.883 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-02-20T13:02:37.959 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-02-20T13:02:38.078 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-02-20T13:02:38.123 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-02-20T13:02:38.200 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-02-20T13:02:38.340 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-02-20T13:02:38.352 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-02-20T13:02:38.493 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-02-20T13:02:38.502 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-02-20T13:02:38.700 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-02-20T13:02:38.711 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-02-20T13:02:38.856 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-02-20T13:02:38.880 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-02-20T13:02:39.024 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-02-20T13:02:39.071 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-02-20T13:02:39.182 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-02-20T13:02:39.276 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-02-20T13:02:39.366 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-02-20T13:02:39.458 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-02-20T13:02:39.557 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-02-20T13:02:39.627 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-02-20T13:02:39.835 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-02-20T13:02:40.105 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2021-02-20T13:02:40.340 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:40 +0000 (0:00:26.593) 0:00:37.990 ***** 2021-02-20T13:02:40.419 INFO:teuthology.task.ansible.out: 2021-02-20T13:02:40.421 INFO:teuthology.task.ansible.out:TASK [users : Create all users without sudo access.] *************************** 2021-02-20T13:02:40.495 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:40 +0000 (0:00:00.153) 0:00:38.144 ***** 2021-02-20T13:02:40.497 INFO:teuthology.task.ansible.out: 2021-02-20T13:02:40.539 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2021-02-20T13:02:40.541 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2021-02-20T13:02:40.712 INFO:teuthology.task.ansible.out: 2021-02-20T13:02:40.714 INFO:teuthology.task.ansible.out:TASK [users : Merge managed_users and managed_admin users] ********************* 2021-02-20T13:02:40.718 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora 2021-02-20T13:02:40.720 INFO:teuthology.task.ansible.out:rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265 2021-02-20T13:02:40.722 INFO:teuthology.task.ansible.out:c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancic 2021-02-20T13:02:40.724 INFO:teuthology.task.ansible.out:h", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1Wi 2021-02-20T13:02:40.726 INFO:teuthology.task.ansible.out:RenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vum 2021-02-20T13:02:40.728 INFO:teuthology.task.ansible.out:rao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69 2021-02-20T13:02:40.730 INFO:teuthology.task.ansible.out:mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 2021-02-20T13:02:40.731 INFO:teuthology.task.ansible.out:5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d8 2021-02-20T13:02:40.732 INFO:teuthology.task.ansible.out:41a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveirada 2021-02-20T13:02:40.734 INFO:teuthology.task.ansible.out:n@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOe 2021-02-20T13:02:40.736 INFO:teuthology.task.ansible.out:nVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9 2021-02-20T13:02:40.739 INFO:teuthology.task.ansible.out:gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": 2021-02-20T13:02:40.741 INFO:teuthology.task.ansible.out:"dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NI 2021-02-20T13:02:40.743 INFO:teuthology.task.ansible.out:Nj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395 2021-02-20T13:02:40.743 INFO:teuthology.task.ansible.out:862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}]}, "changed": false} 2021-02-20T13:02:40.795 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:40 +0000 (0:00:00.301) 0:00:38.445 ***** 2021-02-20T13:02:40.802 INFO:teuthology.task.ansible.out: 2021-02-20T13:02:40.866 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:40 +0000 (0:00:00.070) 0:00:38.515 ***** 2021-02-20T13:02:40.869 INFO:teuthology.task.ansible.out: 2021-02-20T13:02:40.951 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2021-02-20T13:02:40.996 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:40 +0000 (0:00:00.130) 0:00:38.645 ***** 2021-02-20T13:02:41.078 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2021-02-20T13:02:41.079 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora 2021-02-20T13:02:41.081 INFO:teuthology.task.ansible.out:rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265 2021-02-20T13:02:41.083 INFO:teuthology.task.ansible.out:c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancic 2021-02-20T13:02:41.085 INFO:teuthology.task.ansible.out:h", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1Wi 2021-02-20T13:02:41.087 INFO:teuthology.task.ansible.out:RenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vum 2021-02-20T13:02:41.089 INFO:teuthology.task.ansible.out:rao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69 2021-02-20T13:02:41.091 INFO:teuthology.task.ansible.out:mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d8 2021-02-20T13:02:41.093 INFO:teuthology.task.ansible.out:41a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix" 2021-02-20T13:02:41.093 INFO:teuthology.task.ansible.out:, "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveirada 2021-02-20T13:02:41.095 INFO:teuthology.task.ansible.out:n@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOe 2021-02-20T13:02:41.097 INFO:teuthology.task.ansible.out:nVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9 2021-02-20T13:02:41.099 INFO:teuthology.task.ansible.out:gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": 2021-02-20T13:02:41.101 INFO:teuthology.task.ansible.out:"dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NI 2021-02-20T13:02:41.103 INFO:teuthology.task.ansible.out:Nj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}]}, "changed": false} 2021-02-20T13:02:41.173 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:41 +0000 (0:00:00.176) 0:00:38.822 ***** 2021-02-20T13:02:42.694 INFO:teuthology.task.ansible.out: 2021-02-20T13:02:42.696 INFO:teuthology.task.ansible.out:TASK [users : Clone the keys repo] ********************************************* 2021-02-20T13:02:42.696 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com -> localhost] => {"after": "df8946f08e870e2d2565b79099289ca48bd825f3", "attempts": 1, "before": "df8946f08e870e2d2565b79099289ca48bd825f3", "changed": false, "remote_url_changed": false} 2021-02-20T13:02:42.802 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:42 +0000 (0:00:01.629) 0:00:40.452 ***** 2021-02-20T13:02:42.820 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com -> localhost] => {"after": "df8946f08e870e2d2565b79099289ca48bd825f3", "attempts": 1, "before": "df8946f08e870e2d2565b79099289ca48bd825f3", "changed": false, "remote_url_changed": false} 2021-02-20T13:02:42.884 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:02:42 +0000 (0:00:00.081) 0:00:40.534 ***** 2021-02-20T13:02:42.889 INFO:teuthology.task.ansible.out: 2021-02-20T13:02:43.128 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T13:02:43.130 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-02-20T13:02:43.161 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-02-20T13:02:43.322 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-02-20T13:02:43.356 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-02-20T13:02:43.453 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-02-20T13:02:43.520 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-02-20T13:02:43.660 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZ 2021-02-20T13:02:43.662 INFO:teuthology.task.ansible.out:tIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1 2021-02-20T13:02:43.663 INFO:teuthology.task.ansible.out:yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-02-20T13:02:43.845 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-02-20T13:02:43.867 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-02-20T13:02:44.037 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2021-02-20T13:02:44.039 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-02-20T13:02:44.079 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2021-02-20T13:02:44.081 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-02-20T13:02:44.242 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-02-20T13:02:44.274 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-02-20T13:02:44.416 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-02-20T13:02:44.452 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-02-20T13:02:44.589 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-02-20T13:02:44.611 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:02:44.633 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-02-20T13:02:44.678 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:02:44.797 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2021-02-20T13:02:44.800 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-02-20T13:02:44.896 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2021-02-20T13:02:44.897 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-02-20T13:02:44.989 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemer 2021-02-20T13:02:44.992 INFO:teuthology.task.ansible.out:son", "validate_certs": true} 2021-02-20T13:02:45.075 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:02:45.129 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:02:45.131 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemer 2021-02-20T13:02:45.132 INFO:teuthology.task.ansible.out:son", "validate_certs": true} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:02:45.195 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:02:45.202 INFO:teuthology.task.ansible.out: 2021-02-20T13:02:45.299 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-02-20T13:02:45.397 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-02-20T13:02:45.484 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2021-02-20T13:02:45.486 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-02-20T13:02:45.574 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2021-02-20T13:02:45.577 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-02-20T13:02:45.669 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-02-20T13:02:45.818 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-02-20T13:02:45.907 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2021-02-20T13:02:46.005 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2021-02-20T13:02:46.123 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-02-20T13:02:46.177 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-02-20T13:02:46.334 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-02-20T13:02:46.368 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-02-20T13:02:46.554 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-02-20T13:02:46.588 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-02-20T13:02:46.757 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2021-02-20T13:02:46.759 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-02-20T13:02:46.814 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2021-02-20T13:02:46.924 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-02-20T13:02:47.032 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2021-02-20T13:02:47.034 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-02-20T13:02:47.067 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2021-02-20T13:02:47.068 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-02-20T13:02:47.278 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-02-20T13:02:47.291 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-02-20T13:02:47.470 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-02-20T13:02:47.473 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-02-20T13:02:47.645 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46 2021-02-20T13:02:47.647 INFO:teuthology.task.ansible.out:TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-02-20T13:02:47.658 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46 2021-02-20T13:02:47.660 INFO:teuthology.task.ansible.out:TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-02-20T13:02:47.824 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-02-20T13:02:47.825 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-02-20T13:02:48.006 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-02-20T13:02:48.050 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-02-20T13:02:48.208 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-02-20T13:02:48.282 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-02-20T13:02:48.284 INFO:teuthology.task.ansible.out: 2021-02-20T13:02:48.421 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-02-20T13:02:48.477 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-02-20T13:02:48.629 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-02-20T13:02:48.652 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-02-20T13:02:48.789 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2021-02-20T13:02:48.795 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-02-20T13:02:48.831 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2021-02-20T13:02:48.833 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-02-20T13:02:48.994 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-02-20T13:02:49.016 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-02-20T13:02:49.183 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-02-20T13:02:49.194 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-02-20T13:02:49.363 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-02-20T13:02:49.406 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-02-20T13:02:49.555 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-02-20T13:02:49.588 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-02-20T13:02:49.740 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2021-02-20T13:02:49.742 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2021-02-20T13:02:49.874 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-02-20T13:02:49.918 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-02-20T13:02:50.073 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-02-20T13:02:50.092 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-02-20T13:02:50.264 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-02-20T13:02:50.265 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-02-20T13:02:50.489 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-02-20T13:02:50.491 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-02-20T13:02:50.663 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2021-02-20T13:02:50.666 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-02-20T13:02:50.675 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2021-02-20T13:02:50.676 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-02-20T13:02:50.822 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2021-02-20T13:02:50.825 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNK 2021-02-20T13:02:50.827 INFO:teuthology.task.ansible.out:CuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-02-20T13:02:50.962 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-02-20T13:02:51.037 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-02-20T13:02:51.140 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-02-20T13:02:51.208 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-02-20T13:02:51.315 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-02-20T13:02:51.380 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-02-20T13:02:51.551 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2021-02-20T13:02:51.553 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-02-20T13:02:51.561 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2021-02-20T13:02:51.563 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-02-20T13:02:51.724 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2021-02-20T13:02:51.726 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2021-02-20T13:02:51.728 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-02-20T13:02:51.839 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2021-02-20T13:02:51.841 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2021-02-20T13:02:51.843 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-02-20T13:02:51.988 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-02-20T13:02:52.104 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-02-20T13:02:52.200 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-02-20T13:02:52.399 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-02-20T13:02:52.485 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-02-20T13:02:52.636 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-02-20T13:02:52.783 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-02-20T13:02:52.911 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-02-20T13:02:52.996 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-02-20T13:02:53.138 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-02-20T13:02:53.254 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home 2021-02-20T13:02:53.256 INFO:teuthology.task.ansible.out:/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-02-20T13:02:53.328 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home 2021-02-20T13:02:53.330 INFO:teuthology.task.ansible.out:/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-02-20T13:02:53.455 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-02-20T13:02:53.509 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-02-20T13:02:53.647 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-02-20T13:02:53.724 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-02-20T13:02:53.784 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2021-02-20T13:02:53.787 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} 2021-02-20T13:02:53.913 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2021-02-20T13:02:53.915 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} 2021-02-20T13:02:53.975 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-02-20T13:02:54.094 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-02-20T13:02:54.170 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-02-20T13:02:54.303 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-02-20T13:02:54.335 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2021-02-20T13:02:54.337 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-02-20T13:02:54.524 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-02-20T13:02:54.547 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2021-02-20T13:02:54.548 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-02-20T13:02:54.714 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-02-20T13:02:54.747 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-02-20T13:02:54.918 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-02-20T13:02:54.969 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-02-20T13:02:55.113 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-02-20T13:02:55.166 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-02-20T13:02:55.272 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-02-20T13:02:55.336 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-02-20T13:02:55.452 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-02-20T13:02:55.506 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-02-20T13:02:55.601 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-02-20T13:02:55.654 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-02-20T13:02:55.770 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-02-20T13:02:55.824 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-02-20T13:02:55.965 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-02-20T13:02:56.000 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-02-20T13:02:56.104 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-02-20T13:02:56.168 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-02-20T13:02:56.275 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-02-20T13:02:56.334 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-02-20T13:02:56.403 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-02-20T13:02:56.488 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-02-20T13:02:56.554 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-02-20T13:02:56.640 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-02-20T13:02:56.708 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-02-20T13:02:56.815 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-02-20T13:02:56.858 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-02-20T13:02:56.958 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-02-20T13:02:57.013 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-02-20T13:02:57.102 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-02-20T13:02:57.201 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-02-20T13:02:57.276 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-02-20T13:02:57.376 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-02-20T13:02:57.430 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-02-20T13:02:57.537 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-02-20T13:02:57.601 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-02-20T13:02:57.692 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-02-20T13:02:57.788 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-02-20T13:02:57.873 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1 2021-02-20T13:02:57.875 INFO:teuthology.task.ansible.out:lXTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-02-20T13:02:57.947 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-02-20T13:02:58.054 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-02-20T13:02:58.107 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1 2021-02-20T13:02:58.109 INFO:teuthology.task.ansible.out:lXTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-02-20T13:02:58.216 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-02-20T13:02:58.291 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-02-20T13:02:58.454 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-02-20T13:02:58.538 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-02-20T13:02:58.679 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-02-20T13:02:58.826 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-02-20T13:02:58.909 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.locald 2021-02-20T13:02:58.911 INFO:teuthology.task.ansible.out:omain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-02-20T13:02:59.007 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-02-20T13:02:59.125 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2021-02-20T13:02:59.127 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-02-20T13:02:59.231 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.locald 2021-02-20T13:02:59.233 INFO:teuthology.task.ansible.out:omain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-02-20T13:02:59.317 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-02-20T13:02:59.458 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2021-02-20T13:02:59.460 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-02-20T13:02:59.574 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-02-20T13:02:59.693 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-02-20T13:02:59.791 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-02-20T13:02:59.931 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-02-20T13:03:00.020 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-02-20T13:03:00.195 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-02-20T13:03:00.264 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-02-20T13:03:00.434 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-02-20T13:03:00.463 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-02-20T13:03:00.622 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-02-20T13:03:00.701 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-02-20T13:03:00.802 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-02-20T13:03:00.904 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loc 2021-02-20T13:03:00.907 INFO:teuthology.task.ansible.out:al", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-02-20T13:03:01.006 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-02-20T13:03:01.119 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-02-20T13:03:01.224 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loc 2021-02-20T13:03:01.226 INFO:teuthology.task.ansible.out:al", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-02-20T13:03:01.353 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-02-20T13:03:01.408 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-02-20T13:03:01.588 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-02-20T13:03:01.619 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-02-20T13:03:01.726 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-02-20T13:03:01.811 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-02-20T13:03:01.896 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-02-20T13:03:01.971 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-02-20T13:03:02.049 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-02-20T13:03:02.051 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:02.158 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-02-20T13:03:02.223 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2021-02-20T13:03:02.225 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-02-20T13:03:02.372 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-02-20T13:03:02.374 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:02.383 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-02-20T13:03:02.568 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2021-02-20T13:03:02.570 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-02-20T13:03:02.769 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_ 2021-02-20T13:03:02.771 INFO:teuthology.task.ansible.out:certs": true} 2021-02-20T13:03:02.941 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-02-20T13:03:02.953 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-02-20T13:03:03.135 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-02-20T13:03:03.137 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-02-20T13:03:03.298 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-02-20T13:03:03.299 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-02-20T13:03:03.492 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-02-20T13:03:03.514 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-02-20T13:03:03.662 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-02-20T13:03:03.674 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-02-20T13:03:03.812 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-02-20T13:03:03.834 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-02-20T13:03:03.963 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-02-20T13:03:04.012 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-02-20T13:03:04.125 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-02-20T13:03:04.157 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-02-20T13:03:04.257 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-02-20T13:03:04.343 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-02-20T13:03:04.435 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-02-20T13:03:04.541 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-02-20T13:03:04.613 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-02-20T13:03:04.688 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-02-20T13:03:04.825 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-02-20T13:03:04.847 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-02-20T13:03:05.030 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-02-20T13:03:05.032 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-02-20T13:03:05.212 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-02-20T13:03:05.223 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-02-20T13:03:05.372 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-02-20T13:03:05.415 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-02-20T13:03:05.556 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-02-20T13:03:05.557 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjL 2021-02-20T13:03:05.558 INFO:teuthology.task.ansible.out:d+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-02-20T13:03:05.724 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-02-20T13:03:05.736 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-02-20T13:03:05.919 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-02-20T13:03:05.930 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjL 2021-02-20T13:03:05.932 INFO:teuthology.task.ansible.out:d+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-02-20T13:03:06.120 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-02-20T13:03:06.122 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-02-20T13:03:06.286 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-02-20T13:03:06.297 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-02-20T13:03:06.495 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-02-20T13:03:06.517 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-02-20T13:03:06.658 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-02-20T13:03:06.684 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-02-20T13:03:06.827 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-02-20T13:03:06.870 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-02-20T13:03:06.976 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-02-20T13:03:07.035 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-02-20T13:03:07.161 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-02-20T13:03:07.208 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-02-20T13:03:07.325 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-02-20T13:03:07.403 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-02-20T13:03:07.554 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRY 2021-02-20T13:03:07.556 INFO:teuthology.task.ansible.out:VrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-02-20T13:03:07.736 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-02-20T13:03:07.760 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-02-20T13:03:07.877 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-02-20T13:03:07.909 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-02-20T13:03:08.069 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-02-20T13:03:08.071 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-02-20T13:03:08.229 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-02-20T13:03:08.263 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-02-20T13:03:08.397 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-02-20T13:03:08.482 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-02-20T13:03:08.620 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com", "key_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-02-20T13:03:08.653 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-02-20T13:03:08.781 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-02-20T13:03:08.888 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-02-20T13:03:08.952 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:08 +0000 (0:00:26.067) 0:01:06.601 ***** 2021-02-20T13:03:08.996 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com", "key_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-02-20T13:03:09.060 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.062 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.072 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.085 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd 2021-02-20T13:03:09.087 INFO:teuthology.task.ansible.out:0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.096 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.121 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** changed: [smithi183.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-02-20T13:03:09.221 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-02-20T13:03:09.223 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.273 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:09 +0000 (0:00:00.320) 0:01:06.922 ***** 2021-02-20T13:03:09.380 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-02-20T13:03:09.402 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.424 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.436 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.448 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.501 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.523 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.534 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.566 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.588 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.599 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.654 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.656 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-02-20T13:03:09.835 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-02-20T13:03:09.846 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.857 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-02-20T13:03:09.869 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.880 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.906 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.908 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.928 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:09.961 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.006 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.011 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.031 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.047 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.069 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.071 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-02-20T13:03:10.124 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.149 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.184 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.186 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.198 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.219 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.230 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.241 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-02-20T13:03:10.253 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.256 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.265 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.277 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.288 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.290 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.300 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.315 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.326 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.339 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.350 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.362 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.388 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.406 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.408 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.429 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.431 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.441 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.453 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.464 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.476 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.478 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.491 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.503 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.514 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.516 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.525 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.527 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.548 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.549 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.579 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.758 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias 2021-02-20T13:03:10.760 INFO:teuthology.task.ansible.out:", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@lo 2021-02-20T13:03:10.762 INFO:teuthology.task.ansible.out:calhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d 2021-02-20T13:03:10.764 INFO:teuthology.task.ansible.out:832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi183.f 2021-02-20T13:03:10.765 INFO:teuthology.task.ansible.out:ront.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.776 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.778 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.788 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.799 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.811 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.824 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.842 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.875 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.876 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.886 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.897 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.908 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.911 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.931 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.932 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.942 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.953 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.964 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.966 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.976 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.997 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:10.999 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.009 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.013 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:11.027 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.028 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.029 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.041 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.041 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.066 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.067 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.088 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.089 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.099 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.101 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.111 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.122 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.124 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:11.125 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.148 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.149 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:11.149 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.150 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.161 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.163 INFO:teuthology.task.ansible.out: skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.178 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.180 INFO:teuthology.task.ansible.out: skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.190 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.191 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:11.202 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.204 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.214 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.216 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.237 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.238 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:11.249 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.251 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:11.261 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.262 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.272 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.273 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:11.285 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.287 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.296 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.298 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.299 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:11.311 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.313 INFO:teuthology.task.ansible.out: skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.323 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.355 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.366 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.377 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.379 INFO:teuthology.task.ansible.out: skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.390 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.403 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.414 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.425 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.436 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.437 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.447 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.449 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:11.460 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.462 INFO:teuthology.task.ansible.out: skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.463 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.474 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.486 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.498 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.511 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.532 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.543 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.555 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.557 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.567 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.569 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.579 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.604 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.606 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.616 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.618 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:11.630 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.632 INFO:teuthology.task.ansible.out: skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.642 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.654 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.656 INFO:teuthology.task.ansible.out: skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.666 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.668 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.678 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.689 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.691 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.701 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.703 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.713 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.724 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.725 INFO:teuthology.task.ansible.out: skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.738 INFO:teuthology.task.ansible.out: skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.745 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.746 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:11.747 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.748 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.759 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.761 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.771 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.783 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.789 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.795 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.797 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.807 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.819 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.865 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.875 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.944 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:11 +0000 (0:00:02.670) 0:01:09.594 ***** 2021-02-20T13:03:11.961 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.964 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.964 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.965 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.998 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:11.999 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.011 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2021-02-20T13:03:12.013 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.090 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:12 +0000 (0:00:00.143) 0:01:09.737 ***** 2021-02-20T13:03:12.104 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-02-20T13:03:12.106 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.108 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.164 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.166 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:12.209 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.232 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.247 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.286 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.309 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.321 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.352 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.363 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.384 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.406 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.417 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T13:03:12.419 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-02-20T13:03:12.439 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.451 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.473 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.526 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.551 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.572 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.594 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.615 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T13:03:12.618 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-02-20T13:03:12.638 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.648 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.659 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.691 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.724 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.744 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.755 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.809 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.810 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi170.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-02-20T13:03:12.853 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.874 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.900 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.911 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.933 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.947 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.958 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.969 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T13:03:12.972 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.985 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:12.997 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:13.029 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:13.050 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:13.083 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:13.126 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:13.137 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi170.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-02-20T13:03:13.149 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi183.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:13.224 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:13 +0000 (0:00:01.135) 0:01:10.873 ***** 2021-02-20T13:03:13.284 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:13.336 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:13 +0000 (0:00:00.112) 0:01:10.986 ***** 2021-02-20T13:03:13.359 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T13:03:13.364 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-02-20T13:03:13.540 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-02-20T13:03:13.709 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-02-20T13:03:13.877 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-02-20T13:03:13.878 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-02-20T13:03:14.047 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-02-20T13:03:14.058 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-02-20T13:03:14.247 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-02-20T13:03:14.258 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-02-20T13:03:14.448 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-02-20T13:03:14.459 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-02-20T13:03:14.622 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-02-20T13:03:14.623 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-02-20T13:03:14.785 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-02-20T13:03:14.870 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-02-20T13:03:14.986 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-02-20T13:03:15.140 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-02-20T13:03:15.210 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-02-20T13:03:15.361 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-02-20T13:03:15.447 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-02-20T13:03:15.533 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-02-20T13:03:15.646 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-02-20T13:03:15.743 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-02-20T13:03:15.884 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-02-20T13:03:15.992 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-02-20T13:03:16.133 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-02-20T13:03:16.318 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-02-20T13:03:16.372 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-02-20T13:03:16.495 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-02-20T13:03:16.560 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-02-20T13:03:16.710 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-02-20T13:03:16.722 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2021-02-20T13:03:16.883 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-02-20T13:03:16.937 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-02-20T13:03:17.033 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-02-20T13:03:17.120 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-02-20T13:03:17.164 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-02-20T13:03:17.274 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-02-20T13:03:17.349 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-02-20T13:03:17.458 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-02-20T13:03:17.522 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-02-20T13:03:17.622 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-02-20T13:03:17.738 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-02-20T13:03:17.812 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-02-20T13:03:17.908 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2021-02-20T13:03:18.003 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-02-20T13:03:18.082 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-02-20T13:03:18.179 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-02-20T13:03:18.233 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-02-20T13:03:18.368 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-02-20T13:03:18.411 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-02-20T13:03:18.519 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-02-20T13:03:18.563 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-02-20T13:03:18.680 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-02-20T13:03:18.712 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-02-20T13:03:18.856 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-02-20T13:03:18.867 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-02-20T13:03:18.990 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-02-20T13:03:19.013 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-02-20T13:03:19.014 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:19.126 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-02-20T13:03:19.159 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-02-20T13:03:19.297 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-02-20T13:03:19.319 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-02-20T13:03:19.439 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-02-20T13:03:19.482 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-02-20T13:03:19.654 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-02-20T13:03:19.813 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-02-20T13:03:19.932 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "zyan", "name": "zyan", "remove": false, "state": "absent"} 2021-02-20T13:03:20.006 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-02-20T13:03:20.096 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-02-20T13:03:20.184 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-02-20T13:03:20.279 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-02-20T13:03:20.330 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-02-20T13:03:20.439 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-02-20T13:03:20.474 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-02-20T13:03:20.550 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-02-20T13:03:20.677 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-02-20T13:03:21.019 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "zyan", "name": "zyan", "remove": false, "state": "absent"} 2021-02-20T13:03:21.022 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:21.167 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} ok: [smithi170.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-02-20T13:03:21.372 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-02-20T13:03:21.384 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-02-20T13:03:21.551 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-02-20T13:03:21.553 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-02-20T13:03:21.702 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-02-20T13:03:21.765 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-02-20T13:03:21.854 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-02-20T13:03:21.931 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-02-20T13:03:22.047 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:22 +0000 (0:00:08.710) 0:01:19.696 ***** 2021-02-20T13:03:22.071 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-02-20T13:03:22.222 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-02-20T13:03:22.382 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-02-20T13:03:22.554 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-02-20T13:03:22.710 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-02-20T13:03:22.805 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-02-20T13:03:22.807 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "checksum": "5ebc9fcf8859a9c716ab10089a7bec7e44d29368", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "ee06ed012f51c2e3764171d4b9180349", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613826202.1367965-247416619076276/source", "state": "file", "uid": 0} 2021-02-20T13:03:22.898 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:22 +0000 (0:00:00.847) 0:01:20.544 ***** 2021-02-20T13:03:23.010 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:23 +0000 (0:00:00.115) 0:01:20.659 ***** 2021-02-20T13:03:23.037 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-02-20T13:03:23.039 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => { "msg": "Host smithi170.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2021-02-20T13:03:23.113 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:23 +0000 (0:00:00.102) 0:01:20.762 ***** 2021-02-20T13:03:23.252 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-02-20T13:03:23.253 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-02-20T13:03:23.357 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:23 +0000 (0:00:00.244) 0:01:21.006 ***** 2021-02-20T13:03:23.499 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-02-20T13:03:23.501 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:23.560 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:23 +0000 (0:00:00.202) 0:01:21.209 ***** 2021-02-20T13:03:23.652 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2021-02-20T13:03:23.654 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:23.726 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:23 +0000 (0:00:00.162) 0:01:21.372 ***** 2021-02-20T13:03:23.737 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-02-20T13:03:23.737 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"changed": true, "checksum": "5ebc9fcf8859a9c716ab10089a7bec7e44d29368", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "ee06ed012f51c2e3764171d4b9180349", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613826203.0899549-234080172340300/source", "state": "file", "uid": 0} 2021-02-20T13:03:23.796 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:23 +0000 (0:00:00.073) 0:01:21.446 ***** 2021-02-20T13:03:23.806 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-02-20T13:03:23.807 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:23.903 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:23 +0000 (0:00:00.106) 0:01:21.552 ***** 2021-02-20T13:03:23.914 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-02-20T13:03:23.915 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => { "msg": "Host smithi183.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2021-02-20T13:03:23.963 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:23 +0000 (0:00:00.060) 0:01:21.612 ***** 2021-02-20T13:03:23.986 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2021-02-20T13:03:23.988 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:24.053 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:24 +0000 (0:00:00.089) 0:01:21.702 ***** 2021-02-20T13:03:24.077 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-02-20T13:03:24.078 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-02-20T13:03:24.155 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:24 +0000 (0:00:00.102) 0:01:21.805 ***** 2021-02-20T13:03:24.162 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-02-20T13:03:24.163 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:24.242 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:24 +0000 (0:00:00.087) 0:01:21.892 ***** 2021-02-20T13:03:24.244 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:24.263 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-02-20T13:03:24.267 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:24.393 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:24 +0000 (0:00:00.150) 0:01:22.042 ***** 2021-02-20T13:03:24.410 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2021-02-20T13:03:24.413 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:24.466 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:24 +0000 (0:00:00.072) 0:01:22.115 ***** 2021-02-20T13:03:24.477 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:24.569 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:24 +0000 (0:00:00.102) 0:01:22.218 ***** 2021-02-20T13:03:24.579 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:24.581 INFO:teuthology.task.ansible.out:TASK [common : Install yum on Fedora 22 and later] ***************************** 2021-02-20T13:03:24.583 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:24.644 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:24 +0000 (0:00:00.074) 0:01:22.293 ***** 2021-02-20T13:03:24.655 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-02-20T13:03:24.659 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:24.719 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:24 +0000 (0:00:00.076) 0:01:22.369 ***** 2021-02-20T13:03:24.726 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2021-02-20T13:03:24.727 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:24.786 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:24 +0000 (0:00:00.066) 0:01:22.435 ***** 2021-02-20T13:03:24.794 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2021-02-20T13:03:24.795 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:24.856 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:24 +0000 (0:00:00.069) 0:01:22.505 ***** 2021-02-20T13:03:24.880 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-02-20T13:03:24.882 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi170.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T13:03:24.927 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:24 +0000 (0:00:00.070) 0:01:22.576 ***** 2021-02-20T13:03:24.966 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-02-20T13:03:24.968 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:24.970 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:25.013 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:25 +0000 (0:00:00.085) 0:01:22.662 ***** 2021-02-20T13:03:25.020 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:25.083 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:25 +0000 (0:00:00.070) 0:01:22.732 ***** 2021-02-20T13:03:25.091 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2021-02-20T13:03:25.094 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:25.149 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:25 +0000 (0:00:00.065) 0:01:22.798 ***** 2021-02-20T13:03:25.159 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-02-20T13:03:25.161 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:25.212 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:25 +0000 (0:00:00.063) 0:01:22.862 ***** 2021-02-20T13:03:25.223 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2021-02-20T13:03:25.225 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:25.273 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:25 +0000 (0:00:00.060) 0:01:22.922 ***** 2021-02-20T13:03:25.281 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-02-20T13:03:25.283 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T13:03:25.350 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:25 +0000 (0:00:00.076) 0:01:22.999 ***** 2021-02-20T13:03:25.380 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2021-02-20T13:03:25.386 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:25.407 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:25.479 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:25 +0000 (0:00:00.129) 0:01:23.128 ***** 2021-02-20T13:03:25.481 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:25.554 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:25 +0000 (0:00:00.074) 0:01:23.203 ***** 2021-02-20T13:03:25.595 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** skipping: [smithi170.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T13:03:25.668 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:25 +0000 (0:00:00.114) 0:01:23.318 ***** 2021-02-20T13:03:25.680 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-02-20T13:03:25.682 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi183.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T13:03:25.745 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:25 +0000 (0:00:00.076) 0:01:23.395 ***** 2021-02-20T13:03:25.756 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-02-20T13:03:25.758 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:25.822 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:25 +0000 (0:00:00.076) 0:01:23.471 ***** 2021-02-20T13:03:25.833 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2021-02-20T13:03:25.835 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:25.958 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:25 +0000 (0:00:00.135) 0:01:23.607 ***** 2021-02-20T13:03:25.966 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-02-20T13:03:25.968 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:26.058 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:26 +0000 (0:00:00.100) 0:01:23.707 ***** 2021-02-20T13:03:26.067 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-02-20T13:03:26.069 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:26.133 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:26 +0000 (0:00:00.075) 0:01:23.783 ***** 2021-02-20T13:03:26.135 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:26.146 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-02-20T13:03:26.150 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:26.197 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:26 +0000 (0:00:00.063) 0:01:23.846 ***** 2021-02-20T13:03:26.209 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-02-20T13:03:26.210 INFO:teuthology.task.ansible.out: skipping: [smithi183.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T13:03:26.281 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:26 +0000 (0:00:00.084) 0:01:23.931 ***** 2021-02-20T13:03:26.283 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:26.329 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:26.336 INFO:teuthology.task.ansible.out:TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-02-20T13:03:26.339 INFO:teuthology.task.ansible.out: skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:26.405 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:26 +0000 (0:00:00.123) 0:01:24.055 ***** 2021-02-20T13:03:26.429 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:26.430 INFO:teuthology.task.ansible.out:TASK [common : Set rhsm_registered if we're already registered] **************** 2021-02-20T13:03:26.432 INFO:teuthology.task.ansible.out: skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:26.499 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:26 +0000 (0:00:00.093) 0:01:24.149 ***** 2021-02-20T13:03:26.516 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-02-20T13:03:26.522 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T13:03:26.524 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:26.601 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:26 +0000 (0:00:00.102) 0:01:24.251 ***** 2021-02-20T13:03:26.631 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** skipping: [smithi183.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T13:03:26.701 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:26 +0000 (0:00:00.098) 0:01:24.350 ***** 2021-02-20T13:03:26.708 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-02-20T13:03:26.710 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:26.754 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:26 +0000 (0:00:00.052) 0:01:24.403 ***** 2021-02-20T13:03:26.761 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-02-20T13:03:26.763 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:26.831 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:26 +0000 (0:00:00.077) 0:01:24.480 ***** 2021-02-20T13:03:26.833 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:26.842 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2021-02-20T13:03:26.843 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:26.924 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:26 +0000 (0:00:00.093) 0:01:24.574 ***** 2021-02-20T13:03:26.941 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-02-20T13:03:26.943 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:27.017 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:27 +0000 (0:00:00.092) 0:01:24.667 ***** 2021-02-20T13:03:27.041 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-02-20T13:03:27.042 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:27.089 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:27 +0000 (0:00:00.071) 0:01:24.738 ***** 2021-02-20T13:03:27.100 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-02-20T13:03:27.102 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:27.165 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:27 +0000 (0:00:00.076) 0:01:24.814 ***** 2021-02-20T13:03:27.175 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-02-20T13:03:27.178 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:27.225 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:27 +0000 (0:00:00.060) 0:01:24.875 ***** 2021-02-20T13:03:27.235 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:27.237 INFO:teuthology.task.ansible.out:TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:27.287 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:27 +0000 (0:00:00.061) 0:01:24.936 ***** 2021-02-20T13:03:27.289 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:27.297 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:27.299 INFO:teuthology.task.ansible.out:TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-02-20T13:03:27.301 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:27.357 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:27 +0000 (0:00:00.069) 0:01:25.006 ***** 2021-02-20T13:03:27.369 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-02-20T13:03:27.371 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T13:03:27.450 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:27 +0000 (0:00:00.093) 0:01:25.100 ***** 2021-02-20T13:03:27.482 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-02-20T13:03:27.485 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:27.531 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:27 +0000 (0:00:00.080) 0:01:25.181 ***** 2021-02-20T13:03:27.540 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-02-20T13:03:27.542 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:27.595 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:27 +0000 (0:00:00.063) 0:01:25.245 ***** 2021-02-20T13:03:27.603 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-02-20T13:03:27.605 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T13:03:27.668 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:27 +0000 (0:00:00.073) 0:01:25.318 ***** 2021-02-20T13:03:27.676 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2021-02-20T13:03:27.678 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:27.769 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:27 +0000 (0:00:00.100) 0:01:25.419 ***** 2021-02-20T13:03:27.778 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2021-02-20T13:03:27.780 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:27.837 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:27 +0000 (0:00:00.067) 0:01:25.486 ***** 2021-02-20T13:03:27.850 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-02-20T13:03:27.852 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:27.913 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:27 +0000 (0:00:00.075) 0:01:25.562 ***** 2021-02-20T13:03:27.923 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-02-20T13:03:27.940 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:27.983 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:27 +0000 (0:00:00.070) 0:01:25.633 ***** 2021-02-20T13:03:27.992 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-02-20T13:03:27.993 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:28.069 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:28 +0000 (0:00:00.086) 0:01:25.719 ***** 2021-02-20T13:03:28.077 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:28.084 INFO:teuthology.task.ansible.out:TASK [common : Increase the yum timeout.] ************************************** 2021-02-20T13:03:28.086 INFO:teuthology.task.ansible.out: skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:28.162 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:28 +0000 (0:00:00.092) 0:01:25.811 ***** 2021-02-20T13:03:28.164 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:28.174 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:28.175 INFO:teuthology.task.ansible.out:TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:28.241 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:28 +0000 (0:00:00.078) 0:01:25.890 ***** 2021-02-20T13:03:28.253 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:28.256 INFO:teuthology.task.ansible.out:TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-02-20T13:03:28.258 INFO:teuthology.task.ansible.out: skipping: [smithi170.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:28.261 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:28.263 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:28.339 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:28 +0000 (0:00:00.097) 0:01:25.988 ***** 2021-02-20T13:03:28.341 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:28.390 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-02-20T13:03:28.392 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:28.445 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:28 +0000 (0:00:00.106) 0:01:26.094 ***** 2021-02-20T13:03:28.469 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-02-20T13:03:28.471 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-02-20T13:03:28.548 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:28 +0000 (0:00:00.102) 0:01:26.197 ***** 2021-02-20T13:03:28.562 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-02-20T13:03:28.564 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T13:03:28.638 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:28 +0000 (0:00:00.084) 0:01:26.282 ***** 2021-02-20T13:03:28.652 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-02-20T13:03:28.654 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:28.701 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:28 +0000 (0:00:00.068) 0:01:26.350 ***** 2021-02-20T13:03:28.702 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:28.712 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2021-02-20T13:03:28.713 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:28.763 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:28 +0000 (0:00:00.062) 0:01:26.413 ***** 2021-02-20T13:03:28.793 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:28.794 INFO:teuthology.task.ansible.out:TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-02-20T13:03:28.794 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:28.796 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:28.840 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:28 +0000 (0:00:00.076) 0:01:26.489 ***** 2021-02-20T13:03:28.909 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-02-20T13:03:28.911 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.012419", "end": "2021-02-20 13:03:28.651991", "rc": 0, "start": "2021-02-20 13:03:28.639572", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-02-20T13:03:28.981 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:28 +0000 (0:00:00.141) 0:01:26.631 ***** 2021-02-20T13:03:28.998 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2021-02-20T13:03:29.000 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:29.085 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:29 +0000 (0:00:00.103) 0:01:26.735 ***** 2021-02-20T13:03:29.094 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2021-02-20T13:03:29.096 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:29.168 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:29 +0000 (0:00:00.082) 0:01:26.817 ***** 2021-02-20T13:03:29.183 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-02-20T13:03:29.187 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:29.240 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:29 +0000 (0:00:00.072) 0:01:26.889 ***** 2021-02-20T13:03:29.292 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-02-20T13:03:29.294 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:29.361 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:29 +0000 (0:00:00.121) 0:01:27.011 ***** 2021-02-20T13:03:29.371 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-02-20T13:03:29.373 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-02-20T13:03:29.454 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:29 +0000 (0:00:00.093) 0:01:27.104 ***** 2021-02-20T13:03:29.470 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-02-20T13:03:29.471 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "skip_reason": "Conditional result was False"} 2021-02-20T13:03:29.486 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "skip_reason": "Conditional result was False"} 2021-02-20T13:03:29.501 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "skip_reason": "Conditional result was False"} TASK [common : Clean yum cache] ************************************************ skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:29.563 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:29 +0000 (0:00:00.108) 0:01:27.213 ***** 2021-02-20T13:03:29.586 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-02-20T13:03:29.588 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "skip_reason": "Conditional result was False"} 2021-02-20T13:03:29.661 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:29 +0000 (0:00:00.097) 0:01:27.310 ***** 2021-02-20T13:03:29.764 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-02-20T13:03:29.765 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:29.848 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:29 +0000 (0:00:00.187) 0:01:27.498 ***** 2021-02-20T13:03:29.863 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-02-20T13:03:29.865 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.015949", "end": "2021-02-20 13:03:30.083145", "rc": 0, "start": "2021-02-20 13:03:30.067196", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-02-20T13:03:29.990 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:29 +0000 (0:00:00.141) 0:01:27.639 ***** 2021-02-20T13:03:30.004 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2021-02-20T13:03:30.006 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:30.127 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:30 +0000 (0:00:00.136) 0:01:27.776 ***** 2021-02-20T13:03:30.148 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:30.216 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:30 +0000 (0:00:00.089) 0:01:27.865 ***** 2021-02-20T13:03:30.237 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:30.304 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:30 +0000 (0:00:00.088) 0:01:27.954 ***** 2021-02-20T13:03:30.320 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:30.380 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:30 +0000 (0:00:00.076) 0:01:28.030 ***** 2021-02-20T13:03:30.412 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-02-20T13:03:30.413 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:30.539 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:30 +0000 (0:00:00.157) 0:01:28.187 ***** 2021-02-20T13:03:30.561 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-02-20T13:03:30.562 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "skip_reason": "Conditional result was False"} 2021-02-20T13:03:30.564 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "skip_reason": "Conditional result was False"} 2021-02-20T13:03:30.566 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "skip_reason": "Conditional result was False"} 2021-02-20T13:03:30.594 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "skip_reason": "Conditional result was False"} 2021-02-20T13:03:30.734 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:30 +0000 (0:00:00.196) 0:01:28.383 ***** 2021-02-20T13:03:30.753 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2021-02-20T13:03:30.756 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:30.841 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:30 +0000 (0:00:00.107) 0:01:28.491 ***** 2021-02-20T13:03:30.843 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:30.850 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-02-20T13:03:30.852 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:30.895 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:30 +0000 (0:00:00.053) 0:01:28.544 ***** 2021-02-20T13:03:30.977 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2021-02-20T13:03:30.979 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:31.074 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:31 +0000 (0:00:00.174) 0:01:28.719 ***** 2021-02-20T13:03:31.124 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2021-02-20T13:03:31.127 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:31.163 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:31 +0000 (0:00:00.091) 0:01:28.810 ***** 2021-02-20T13:03:31.222 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-02-20T13:03:31.225 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:31.297 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:31 +0000 (0:00:00.136) 0:01:28.947 ***** 2021-02-20T13:03:31.397 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:31.434 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:31 +0000 (0:00:00.137) 0:01:29.084 ***** 2021-02-20T13:03:31.745 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-02-20T13:03:31.747 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.738140", "end": "2021-02-20 13:03:31.500443", "rc": 0, "start": "2021-02-20 13:03:30.762303", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:03:31.826 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:31 +0000 (0:00:00.391) 0:01:29.476 ***** 2021-02-20T13:03:31.904 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:31.946 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:31 +0000 (0:00:00.119) 0:01:29.595 ***** 2021-02-20T13:03:32.163 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-02-20T13:03:32.165 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the apt module rather than running 'apt-get'. If you need to use command because apt is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.053497", "end": "2021-02-20 13:03:31.894688", "rc": 0, "start": "2021-02-20 13:03:31.841191", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:03:32.213 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:32 +0000 (0:00:00.266) 0:01:29.862 ***** 2021-02-20T13:03:32.241 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-02-20T13:03:32.243 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.625384", "end": "2021-02-20 13:03:32.500406", "rc": 0, "start": "2021-02-20 13:03:31.875022", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:03:32.295 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:32 +0000 (0:00:00.082) 0:01:29.944 ***** 2021-02-20T13:03:32.352 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:32.393 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:32 +0000 (0:00:00.098) 0:01:30.043 ***** 2021-02-20T13:03:32.627 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-02-20T13:03:32.629 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.046126", "end": "2021-02-20 13:03:32.890163", "rc": 0, "start": "2021-02-20 13:03:32.844037", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:03:32.676 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:32 +0000 (0:00:00.283) 0:01:30.326 ***** 2021-02-20T13:03:36.280 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-02-20T13:03:36.282 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1613825899, "cache_updated": false, "changed": false} 2021-02-20T13:03:36.388 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:36 +0000 (0:00:03.711) 0:01:34.038 ***** 2021-02-20T13:03:36.450 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1613825943, "cache_updated": false, "changed": false} 2021-02-20T13:03:36.509 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:36 +0000 (0:00:00.120) 0:01:34.158 ***** 2021-02-20T13:03:37.316 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-02-20T13:03:37.318 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"cache_update_time": 1613825899, "cache_updated": false, "changed": false} 2021-02-20T13:03:37.405 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:37 +0000 (0:00:00.896) 0:01:35.055 ***** 2021-02-20T13:03:37.423 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"cache_update_time": 1613825943, "cache_updated": false, "changed": false} 2021-02-20T13:03:37.526 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:37 +0000 (0:00:00.120) 0:01:35.175 ***** 2021-02-20T13:03:37.532 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-02-20T13:03:37.534 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:37.624 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:37 +0000 (0:00:00.098) 0:01:35.273 ***** 2021-02-20T13:03:37.645 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:37.712 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:37 +0000 (0:00:00.088) 0:01:35.362 ***** 2021-02-20T13:03:38.767 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2021-02-20T13:03:38.769 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-02-20T13:03:38.818 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:38 +0000 (0:00:01.105) 0:01:36.467 ***** 2021-02-20T13:03:38.850 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-02-20T13:03:38.905 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:38 +0000 (0:00:00.087) 0:01:36.555 ***** 2021-02-20T13:03:39.963 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2021-02-20T13:03:39.965 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"cache_update_time": 1613825943, "cache_updated": false, "changed": false} ok: [smithi170.front.sepia.ceph.com] => {"cache_update_time": 1613825899, "cache_updated": false, "changed": false} 2021-02-20T13:03:40.124 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:40 +0000 (0:00:01.218) 0:01:37.773 ***** 2021-02-20T13:03:40.261 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:40 +0000 (0:00:00.136) 0:01:37.910 ***** 2021-02-20T13:03:40.741 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-02-20T13:03:40.743 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-02-20T13:03:40.860 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-02-20T13:03:41.245 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-02-20T13:03:41.320 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-02-20T13:03:41.683 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-02-20T13:03:41.740 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:41 +0000 (0:00:01.479) 0:01:39.389 ***** 2021-02-20T13:03:41.743 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:41.837 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-02-20T13:03:41.915 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:41 +0000 (0:00:00.175) 0:01:39.565 ***** 2021-02-20T13:03:42.004 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-02-20T13:03:42.006 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T13:03:42.087 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:42 +0000 (0:00:00.171) 0:01:39.736 ***** 2021-02-20T13:03:42.117 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T13:03:42.176 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:42 +0000 (0:00:00.089) 0:01:39.826 ***** 2021-02-20T13:03:42.585 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:42.587 INFO:teuthology.task.ansible.out:TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-02-20T13:03:42.588 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-02-20T13:03:42.737 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-02-20T13:03:43.063 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-02-20T13:03:43.130 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-02-20T13:03:43.456 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2021-02-20T13:03:43.629 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:43 +0000 (0:00:01.452) 0:01:41.278 ***** 2021-02-20T13:03:43.660 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2021-02-20T13:03:43.769 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:43 +0000 (0:00:00.140) 0:01:41.419 ***** 2021-02-20T13:03:43.795 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-02-20T13:03:43.797 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-02-20T13:03:43.850 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:43 +0000 (0:00:00.080) 0:01:41.500 ***** 2021-02-20T13:03:43.873 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-02-20T13:03:43.992 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:43 +0000 (0:00:00.142) 0:01:41.642 ***** 2021-02-20T13:03:44.023 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2021-02-20T13:03:44.026 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:44.108 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:44 +0000 (0:00:00.115) 0:01:41.758 ***** 2021-02-20T13:03:44.123 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:44.245 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:44 +0000 (0:00:00.136) 0:01:41.895 ***** 2021-02-20T13:03:44.261 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2021-02-20T13:03:44.619 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T13:03:44.734 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:44 +0000 (0:00:00.488) 0:01:42.383 ***** 2021-02-20T13:03:44.742 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T13:03:44.857 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:44 +0000 (0:00:00.122) 0:01:42.506 ***** 2021-02-20T13:03:44.900 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-02-20T13:03:44.902 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-20T13:03:44.913 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-20T13:03:44.982 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:44 +0000 (0:00:00.124) 0:01:42.631 ***** 2021-02-20T13:03:45.032 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-20T13:03:45.045 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-20T13:03:45.056 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-20T13:03:45.078 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-02-20T13:03:45.080 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install nrpe without epel] ************************************** skipping: [smithi183.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-20T13:03:45.199 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:45 +0000 (0:00:00.217) 0:01:42.849 ***** 2021-02-20T13:03:45.275 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:45 +0000 (0:00:00.075) 0:01:42.924 ***** 2021-02-20T13:03:45.350 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-02-20T13:03:45.352 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:45.394 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:45 +0000 (0:00:00.118) 0:01:43.043 ***** 2021-02-20T13:03:46.183 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** ok: [smithi170.front.sepia.ceph.com] => {"cache_update_time": 1613825899, "cache_updated": false, "changed": false} 2021-02-20T13:03:46.275 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:46 +0000 (0:00:00.881) 0:01:43.924 ***** 2021-02-20T13:03:46.487 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"cache_update_time": 1613825943, "cache_updated": false, "changed": false} 2021-02-20T13:03:46.540 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:46 +0000 (0:00:00.265) 0:01:44.190 ***** 2021-02-20T13:03:46.768 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2021-02-20T13:03:46.773 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-02-20T13:03:46.842 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:46 +0000 (0:00:00.301) 0:01:44.491 ***** 2021-02-20T13:03:47.110 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-02-20T13:03:47.183 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:47 +0000 (0:00:00.340) 0:01:44.832 ***** 2021-02-20T13:03:47.358 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-02-20T13:03:47.436 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:47 +0000 (0:00:00.253) 0:01:45.085 ***** 2021-02-20T13:03:47.730 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-02-20T13:03:47.732 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-02-20T13:03:47.879 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-02-20T13:03:47.882 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-02-20T13:03:47.979 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:47 +0000 (0:00:00.543) 0:01:45.629 ***** 2021-02-20T13:03:47.999 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:48.007 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:48.026 INFO:teuthology.task.ansible.out:TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ ok: [smithi170.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-02-20T13:03:48.140 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:48 +0000 (0:00:00.146) 0:01:45.775 ***** 2021-02-20T13:03:48.264 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-02-20T13:03:48.272 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:48.322 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:48 +0000 (0:00:00.196) 0:01:45.972 ***** 2021-02-20T13:03:48.333 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-02-20T13:03:48.337 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-02-20T13:03:48.401 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-02-20T13:03:48.495 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:48 +0000 (0:00:00.172) 0:01:46.145 ***** 2021-02-20T13:03:48.512 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2021-02-20T13:03:48.514 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T13:03:48.683 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:48 +0000 (0:00:00.187) 0:01:46.332 ***** 2021-02-20T13:03:48.702 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-02-20T13:03:48.704 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:48.782 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:48 +0000 (0:00:00.099) 0:01:46.432 ***** 2021-02-20T13:03:48.800 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-02-20T13:03:48.802 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:48.894 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:48 +0000 (0:00:00.111) 0:01:46.543 ***** 2021-02-20T13:03:48.916 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** skipping: [smithi183.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T13:03:48.922 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:48.978 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:48 +0000 (0:00:00.083) 0:01:46.627 ***** 2021-02-20T13:03:49.132 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-02-20T13:03:49.134 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:49.237 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:49 +0000 (0:00:00.258) 0:01:46.886 ***** 2021-02-20T13:03:49.238 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:49.570 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-02-20T13:03:49.572 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-02-20T13:03:49.633 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:49 +0000 (0:00:00.396) 0:01:47.283 ***** 2021-02-20T13:03:49.906 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-02-20T13:03:49.957 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:49 +0000 (0:00:00.323) 0:01:47.607 ***** 2021-02-20T13:03:50.611 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-02-20 13:01:41 UTC", "ActiveEnterTimestampMonotonic": "22254406", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "remote-fs.target time-sync.target local-fs.target sysinit.target systemd-journald.socket systemd-tmpfiles-setup.service network.target -.mount basic.target system.slice nss-lookup.target var-run.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-02-20 13:01:41 UTC", "AssertTimestampMonotonic": "22253518", "Before": "plymouth-quit.service xdm.service getty@tty1.service multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-02-20 13:01:41 UTC", "ConditionTimestampMonotonic": "22253518", "ConfigurationDirectoryMode": "0755", "Confl 2021-02-20T13:03:50.612 INFO:teuthology.task.ansible.out:icts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-server.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1283", "ExecMainStartTimestamp": "Sat 2021-02-20 13:01:41 UTC", "ExecMainStartTimestampMonotonic": "22254370", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sat 2021-02-20 13:01:41 UTC] ; stop_time=[n/a] ; pid=1283 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-02-20 13:01:41 UTC", "InactiveExitTimestampMonotonic": "22254406", "InvocationID": "1efd84cf626a4a908165ca45215855c7", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "Key 2021-02-20T13:03:50.614 INFO:teuthology.task.ansible.out:ringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1283", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount system.slice sysinit.target", 2021-02-20T13:03:50.616 INFO:teuthology.task.ansible.out:"RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 13:01:41 UTC", "StateChangeTimestampMonotonic": "22254406", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-02-20 13:01:41 UTC", "WatchdogTimestampMonotonic": "22254405", "WatchdogUSec": "0"}} 2021-02-20T13:03:50.674 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:50 +0000 (0:00:00.716) 0:01:48.324 ***** 2021-02-20T13:03:50.692 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-02-20 13:01:31 UTC", "ActiveEnterTimestampMonotonic": "18910354", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "var-run.mount network.target systemd-tmpfiles-setup.service remote-fs.target system.slice time-sync.target systemd-journald.socket -.mount local-fs.target basic.target sysinit.target nss-lookup.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-02-20 13:01:31 UTC", "AssertTimestampMonotonic": "18909707", "Before": "xdm.service getty@tty1.service shutdown.target multi-user.target plymouth-quit.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-02-20 13:01:31 UTC", "ConditionTimestampMonotonic": "18909707", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-ser 2021-02-20T13:03:50.694 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1306", "ExecMainStartTimestamp": "Sat 2021-02-20 13:01:31 UTC", "ExecMainStartTimestampMonotonic": "18910326", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sat 2021-02-20 13:01:31 UTC] ; stop_time=[n/a] ; pid=1306 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-02-20 13:01:31 UTC", "InactiveExitTimestampMonotonic": "18910354", "InvocationID": "56bc1b763c4c4e0881ba41a49a48089f", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "i 2021-02-20T13:03:50.696 INFO:teuthology.task.ansible.out:nfinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1306", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2021-02-20T13:03:50.697 INFO:teuthology.task.ansible.out: "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 13:01:31 UTC", "StateChangeTimestampMonotonic": "18910354", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-02-20 13:01:31 UTC", "WatchdogTimestampMonotonic": "18910353", "WatchdogUSec": "0"}} 2021-02-20T13:03:50.739 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:50 +0000 (0:00:00.064) 0:01:48.389 ***** 2021-02-20T13:03:50.753 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-02-20T13:03:50.754 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:50.819 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:50 +0000 (0:00:00.079) 0:01:48.468 ***** 2021-02-20T13:03:50.849 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:50.913 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:50 +0000 (0:00:00.093) 0:01:48.562 ***** 2021-02-20T13:03:50.932 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-02-20T13:03:50.934 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:51.005 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:50 +0000 (0:00:00.092) 0:01:48.654 ***** 2021-02-20T13:03:51.024 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:51.078 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:51 +0000 (0:00:00.071) 0:01:48.726 ***** 2021-02-20T13:03:51.088 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2021-02-20T13:03:51.090 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:51.168 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:51 +0000 (0:00:00.091) 0:01:48.817 ***** 2021-02-20T13:03:51.193 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:51.246 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:51 +0000 (0:00:00.078) 0:01:48.896 ***** 2021-02-20T13:03:51.269 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-02-20T13:03:51.271 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:51.334 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:51 +0000 (0:00:00.087) 0:01:48.983 ***** 2021-02-20T13:03:51.353 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:51.434 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:51 +0000 (0:00:00.100) 0:01:49.084 ***** 2021-02-20T13:03:51.447 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-02-20T13:03:51.450 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:51.529 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:51 +0000 (0:00:00.094) 0:01:49.178 ***** 2021-02-20T13:03:51.543 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:51.630 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:51 +0000 (0:00:00.094) 0:01:49.273 ***** 2021-02-20T13:03:51.635 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2021-02-20T13:03:51.636 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:51.702 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:51 +0000 (0:00:00.077) 0:01:49.351 ***** 2021-02-20T13:03:51.710 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:51.776 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:51 +0000 (0:00:00.074) 0:01:49.425 ***** 2021-02-20T13:03:51.785 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-02-20T13:03:51.786 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:51.863 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:51 +0000 (0:00:00.087) 0:01:49.512 ***** 2021-02-20T13:03:51.888 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:51.936 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:51 +0000 (0:00:00.073) 0:01:49.586 ***** 2021-02-20T13:03:51.956 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-02-20T13:03:51.958 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:52.027 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:52 +0000 (0:00:00.090) 0:01:49.676 ***** 2021-02-20T13:03:52.042 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:52.122 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:52 +0000 (0:00:00.094) 0:01:49.772 ***** 2021-02-20T13:03:52.129 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-02-20T13:03:52.131 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:52.184 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:52 +0000 (0:00:00.061) 0:01:49.833 ***** 2021-02-20T13:03:52.204 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:52.255 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:52 +0000 (0:00:00.071) 0:01:49.904 ***** 2021-02-20T13:03:52.264 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:52.337 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:52 +0000 (0:00:00.082) 0:01:49.986 ***** 2021-02-20T13:03:52.352 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:52.426 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:52 +0000 (0:00:00.088) 0:01:50.075 ***** 2021-02-20T13:03:52.436 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-02-20T13:03:52.438 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-02-20T13:03:52.500 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:52 +0000 (0:00:00.074) 0:01:50.149 ***** 2021-02-20T13:03:52.510 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-02-20T13:03:52.557 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:52 +0000 (0:00:00.056) 0:01:50.206 ***** 2021-02-20T13:03:52.599 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-02-20T13:03:52.601 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-02-20T13:03:52.603 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-02-20T13:03:52.672 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:52 +0000 (0:00:00.111) 0:01:50.317 ***** 2021-02-20T13:03:52.719 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:52 +0000 (0:00:00.050) 0:01:50.368 ***** 2021-02-20T13:03:52.752 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-02-20T13:03:52.754 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2021-02-20T13:03:52.802 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:52 +0000 (0:00:00.079) 0:01:50.447 ***** 2021-02-20T13:03:52.806 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2021-02-20T13:03:52.848 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:52 +0000 (0:00:00.050) 0:01:50.497 ***** 2021-02-20T13:03:52.869 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-02-20T13:03:52.871 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-02-20T13:03:52.925 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:52 +0000 (0:00:00.077) 0:01:50.575 ***** 2021-02-20T13:03:52.926 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:52.942 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-02-20T13:03:52.944 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:53.015 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:53 +0000 (0:00:00.089) 0:01:50.664 ***** 2021-02-20T13:03:53.174 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:53.176 INFO:teuthology.task.ansible.out:TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-02-20T13:03:53.229 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:53 +0000 (0:00:00.214) 0:01:50.879 ***** 2021-02-20T13:03:53.237 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-02-20T13:03:53.283 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:53 +0000 (0:00:00.053) 0:01:50.933 ***** 2021-02-20T13:03:53.366 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-02-20T13:03:53.367 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} 2021-02-20T13:03:53.409 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:53 +0000 (0:00:00.125) 0:01:51.058 ***** 2021-02-20T13:03:53.447 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-02-20T13:03:53.449 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:53.483 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:53 +0000 (0:00:00.074) 0:01:51.132 ***** 2021-02-20T13:03:53.490 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-02-20T13:03:53.491 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} 2021-02-20T13:03:53.539 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:53 +0000 (0:00:00.056) 0:01:51.189 ***** 2021-02-20T13:03:53.547 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-02-20T13:03:53.549 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:53.589 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:53 +0000 (0:00:00.049) 0:01:51.238 ***** 2021-02-20T13:03:53.597 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-02-20T13:03:53.598 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:53.642 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:53 +0000 (0:00:00.052) 0:01:51.291 ***** 2021-02-20T13:03:53.670 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-02-20T13:03:53.672 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:53.709 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:53 +0000 (0:00:00.067) 0:01:51.358 ***** 2021-02-20T13:03:53.770 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** ok: [smithi170.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-02-20T13:03:53.810 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:53 +0000 (0:00:00.101) 0:01:51.460 ***** 2021-02-20T13:03:53.887 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-02-20T13:03:53.930 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:53 +0000 (0:00:00.120) 0:01:51.580 ***** 2021-02-20T13:03:54.339 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-02-20T13:03:54.340 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-02-20T13:03:54.381 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:54 +0000 (0:00:00.450) 0:01:52.030 ***** 2021-02-20T13:03:54.430 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi170"}, "changed": false} 2021-02-20T13:03:54.469 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:54 +0000 (0:00:00.088) 0:01:52.118 ***** 2021-02-20T13:03:54.479 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-02-20T13:03:54.481 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-02-20T13:03:54.524 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:54 +0000 (0:00:00.055) 0:01:52.173 ***** 2021-02-20T13:03:54.586 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-02-20T13:03:54.588 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi183"}, "changed": false} 2021-02-20T13:03:54.633 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:54 +0000 (0:00:00.109) 0:01:52.282 ***** 2021-02-20T13:03:55.679 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ changed: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi170.front.sepia.ceph.com", "ansible_hostname": "smithi170", "ansible_nodename": "smithi170"}, "changed": true, "name": "smithi170"} 2021-02-20T13:03:55.735 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:55 +0000 (0:00:01.101) 0:01:53.384 ***** 2021-02-20T13:03:55.779 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi183.front.sepia.ceph.com", "ansible_hostname": "smithi183", "ansible_nodename": "smithi183"}, "changed": true, "name": "smithi183"} 2021-02-20T13:03:55.817 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:55 +0000 (0:00:00.081) 0:01:53.466 ***** 2021-02-20T13:03:56.204 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2021-02-20T13:03:56.206 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-02-20T13:03:56.253 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:56 +0000 (0:00:00.436) 0:01:53.902 ***** 2021-02-20T13:03:56.270 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-02-20T13:03:56.311 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:56 +0000 (0:00:00.058) 0:01:53.961 ***** 2021-02-20T13:03:56.313 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:56.591 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-02-20T13:03:56.632 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:56 +0000 (0:00:00.320) 0:01:54.281 ***** 2021-02-20T13:03:56.635 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:56.691 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-02-20T13:03:56.745 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:56 +0000 (0:00:00.113) 0:01:54.395 ***** 2021-02-20T13:03:57.267 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** changed: [smithi170.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-02-20T13:03:57.340 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:57 +0000 (0:00:00.594) 0:01:54.989 ***** 2021-02-20T13:03:57.368 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-02-20T13:03:57.423 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:57 +0000 (0:00:00.083) 0:01:55.072 ***** 2021-02-20T13:03:57.432 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-02-20T13:03:57.434 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:57.501 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:57 +0000 (0:00:00.077) 0:01:55.150 ***** 2021-02-20T13:03:57.517 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:57.581 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:57 +0000 (0:00:00.079) 0:01:55.230 ***** 2021-02-20T13:03:57.596 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-02-20T13:03:57.603 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:57.661 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:57 +0000 (0:00:00.080) 0:01:55.310 ***** 2021-02-20T13:03:57.670 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:57.742 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:57 +0000 (0:00:00.080) 0:01:55.391 ***** 2021-02-20T13:03:57.748 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-02-20T13:03:57.750 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:57.796 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:57 +0000 (0:00:00.054) 0:01:55.446 ***** 2021-02-20T13:03:57.815 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:57.855 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:57 +0000 (0:00:00.058) 0:01:55.504 ***** 2021-02-20T13:03:57.862 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-02-20T13:03:57.864 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:57.902 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:57 +0000 (0:00:00.047) 0:01:55.551 ***** 2021-02-20T13:03:57.928 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:57.972 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:57 +0000 (0:00:00.069) 0:01:55.621 ***** 2021-02-20T13:03:57.978 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-02-20T13:03:57.980 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:58.020 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:58 +0000 (0:00:00.047) 0:01:55.669 ***** 2021-02-20T13:03:58.030 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:58.074 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:58 +0000 (0:00:00.054) 0:01:55.723 ***** 2021-02-20T13:03:58.086 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:58.147 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:58 +0000 (0:00:00.073) 0:01:55.797 ***** 2021-02-20T13:03:58.157 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:58.227 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:58 +0000 (0:00:00.079) 0:01:55.877 ***** 2021-02-20T13:03:58.245 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-02-20T13:03:58.247 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:58.293 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:58 +0000 (0:00:00.065) 0:01:55.942 ***** 2021-02-20T13:03:58.302 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:58.341 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:58 +0000 (0:00:00.048) 0:01:55.991 ***** 2021-02-20T13:03:58.354 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-02-20T13:03:58.356 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:58.401 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:58 +0000 (0:00:00.059) 0:01:56.051 ***** 2021-02-20T13:03:58.411 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:58.456 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:58 +0000 (0:00:00.055) 0:01:56.106 ***** 2021-02-20T13:03:58.464 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-02-20T13:03:58.533 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:58 +0000 (0:00:00.076) 0:01:56.183 ***** 2021-02-20T13:03:58.638 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:58 +0000 (0:00:00.104) 0:01:56.287 ***** 2021-02-20T13:03:58.667 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-02-20T13:03:58.719 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:58 +0000 (0:00:00.080) 0:01:56.368 ***** 2021-02-20T13:03:58.780 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:58 +0000 (0:00:00.061) 0:01:56.429 ***** 2021-02-20T13:03:58.782 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:58.790 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:58.792 INFO:teuthology.task.ansible.out:TASK [testnode : Enable copr repos] ******************************************** 2021-02-20T13:03:58.847 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:58 +0000 (0:00:00.067) 0:01:56.497 ***** 2021-02-20T13:03:58.849 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:58.917 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:58 +0000 (0:00:00.070) 0:01:56.567 ***** 2021-02-20T13:03:58.925 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:58.971 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:58 +0000 (0:00:00.053) 0:01:56.620 ***** 2021-02-20T13:03:58.982 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:59.056 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:59 +0000 (0:00:00.085) 0:01:56.706 ***** 2021-02-20T13:03:59.058 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:59.090 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-02-20T13:03:59.094 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-02-20T13:03:59.146 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:59 +0000 (0:00:00.089) 0:01:56.795 ***** 2021-02-20T13:03:59.154 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2021-02-20T13:03:59.165 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-02-20T13:03:59.226 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:59 +0000 (0:00:00.079) 0:01:56.875 ***** 2021-02-20T13:03:59.241 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-02-20T13:03:59.243 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:59.297 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:59 +0000 (0:00:00.071) 0:01:56.946 ***** 2021-02-20T13:03:59.310 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:59.365 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:59 +0000 (0:00:00.068) 0:01:57.014 ***** 2021-02-20T13:03:59.403 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T13:03:59.406 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:59.502 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:59 +0000 (0:00:00.137) 0:01:57.152 ***** 2021-02-20T13:03:59.554 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:59.561 INFO:teuthology.task.ansible.out: 2021-02-20T13:03:59.649 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:59 +0000 (0:00:00.147) 0:01:57.299 ***** 2021-02-20T13:03:59.670 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-02-20T13:03:59.672 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:59.746 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:59 +0000 (0:00:00.095) 0:01:57.395 ***** 2021-02-20T13:03:59.754 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:59.800 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:59 +0000 (0:00:00.055) 0:01:57.450 ***** 2021-02-20T13:03:59.809 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-02-20T13:03:59.811 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:59.859 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:59 +0000 (0:00:00.057) 0:01:57.508 ***** 2021-02-20T13:03:59.882 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:03:59.926 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:59 +0000 (0:00:00.067) 0:01:57.575 ***** 2021-02-20T13:03:59.964 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-02-20T13:03:59.966 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:00.003 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:03:59 +0000 (0:00:00.077) 0:01:57.652 ***** 2021-02-20T13:04:00.033 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:00.105 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:00 +0000 (0:00:00.101) 0:01:57.754 ***** 2021-02-20T13:04:00.114 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-02-20T13:04:00.115 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:00.174 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:00 +0000 (0:00:00.068) 0:01:57.823 ***** 2021-02-20T13:04:00.218 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:00.256 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:00 +0000 (0:00:00.082) 0:01:57.905 ***** 2021-02-20T13:04:00.263 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-02-20T13:04:00.265 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:00.316 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:00 +0000 (0:00:00.059) 0:01:57.965 ***** 2021-02-20T13:04:00.324 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:00.418 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:00 +0000 (0:00:00.102) 0:01:58.068 ***** 2021-02-20T13:04:00.427 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-02-20T13:04:00.428 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:00.484 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:00 +0000 (0:00:00.065) 0:01:58.133 ***** 2021-02-20T13:04:00.507 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:00.584 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:00 +0000 (0:00:00.099) 0:01:58.233 ***** 2021-02-20T13:04:00.609 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-02-20T13:04:00.610 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:00.704 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:00 +0000 (0:00:00.120) 0:01:58.354 ***** 2021-02-20T13:04:00.717 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:00.774 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:00 +0000 (0:00:00.070) 0:01:58.424 ***** 2021-02-20T13:04:00.819 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-02-20T13:04:00.821 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:00.872 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:00 +0000 (0:00:00.097) 0:01:58.521 ***** 2021-02-20T13:04:00.880 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:00.933 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:00 +0000 (0:00:00.061) 0:01:58.583 ***** 2021-02-20T13:04:00.945 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-02-20T13:04:00.946 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:01.001 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:00 +0000 (0:00:00.067) 0:01:58.650 ***** 2021-02-20T13:04:01.032 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:01.083 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:01 +0000 (0:00:00.082) 0:01:58.732 ***** 2021-02-20T13:04:01.085 INFO:teuthology.task.ansible.out: 2021-02-20T13:04:01.092 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-02-20T13:04:01.093 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:01.158 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:01 +0000 (0:00:00.074) 0:01:58.807 ***** 2021-02-20T13:04:01.159 INFO:teuthology.task.ansible.out: 2021-02-20T13:04:01.181 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:01.255 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:01 +0000 (0:00:00.097) 0:01:58.904 ***** 2021-02-20T13:04:01.279 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:01.334 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:01 +0000 (0:00:00.079) 0:01:58.983 ***** 2021-02-20T13:04:01.347 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:01.398 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:01 +0000 (0:00:00.059) 0:01:59.043 ***** 2021-02-20T13:04:01.406 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-02-20T13:04:01.408 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:01.449 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:01 +0000 (0:00:00.054) 0:01:59.098 ***** 2021-02-20T13:04:01.456 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:01.500 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:01 +0000 (0:00:00.051) 0:01:59.150 ***** 2021-02-20T13:04:01.510 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-02-20T13:04:01.512 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:01.603 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:01 +0000 (0:00:00.102) 0:01:59.253 ***** 2021-02-20T13:04:01.635 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:01.688 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:01 +0000 (0:00:00.084) 0:01:59.337 ***** 2021-02-20T13:04:01.704 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2021-02-20T13:04:01.706 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:01.750 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:01 +0000 (0:00:00.061) 0:01:59.399 ***** 2021-02-20T13:04:01.758 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:01.820 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:01 +0000 (0:00:00.070) 0:01:59.469 ***** 2021-02-20T13:04:01.840 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-02-20T13:04:01.842 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:01.929 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:01 +0000 (0:00:00.109) 0:01:59.579 ***** 2021-02-20T13:04:01.940 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:02.002 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:01 +0000 (0:00:00.072) 0:01:59.652 ***** 2021-02-20T13:04:02.015 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:02.072 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:02 +0000 (0:00:00.069) 0:01:59.721 ***** 2021-02-20T13:04:02.084 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:02.163 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:02 +0000 (0:00:00.088) 0:01:59.810 ***** 2021-02-20T13:04:02.177 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-02-20T13:04:02.179 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:02.219 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:02 +0000 (0:00:00.058) 0:01:59.868 ***** 2021-02-20T13:04:02.229 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:02.273 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:02 +0000 (0:00:00.054) 0:01:59.922 ***** 2021-02-20T13:04:02.281 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-02-20T13:04:02.283 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:02.284 INFO:teuthology.task.ansible.out: 2021-02-20T13:04:02.334 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:02 +0000 (0:00:00.061) 0:01:59.983 ***** 2021-02-20T13:04:02.342 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:02.387 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:02 +0000 (0:00:00.050) 0:02:00.034 ***** 2021-02-20T13:04:02.540 INFO:teuthology.task.ansible.out: 2021-02-20T13:04:02.542 INFO:teuthology.task.ansible.out:TASK [testnode : Check for custom repos] *************************************** 2021-02-20T13:04:02.542 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005186", "end": "2021-02-20 13:04:02.785115", "rc": 0, "start": "2021-02-20 13:04:02.779929", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:04:02.596 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:02 +0000 (0:00:00.211) 0:02:00.245 ***** 2021-02-20T13:04:02.610 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005201", "end": "2021-02-20 13:04:02.351258", "rc": 0, "start": "2021-02-20 13:04:02.346057", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:04:02.671 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:02 +0000 (0:00:00.075) 0:02:00.321 ***** 2021-02-20T13:04:02.679 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-02-20T13:04:02.726 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:02 +0000 (0:00:00.054) 0:02:00.375 ***** 2021-02-20T13:04:02.782 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:02 +0000 (0:00:00.055) 0:02:00.431 ***** 2021-02-20T13:04:03.107 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-02-20T13:04:03.108 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-02-20T13:04:03.153 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:03 +0000 (0:00:00.371) 0:02:00.803 ***** 2021-02-20T13:04:03.162 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-02-20T13:04:03.215 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:03 +0000 (0:00:00.061) 0:02:00.864 ***** 2021-02-20T13:04:03.224 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:03.263 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:03 +0000 (0:00:00.047) 0:02:00.912 ***** 2021-02-20T13:04:03.271 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:03.313 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:03 +0000 (0:00:00.050) 0:02:00.963 ***** 2021-02-20T13:04:04.602 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-02-20T13:04:04.604 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} ok: [smithi170.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-02-20T13:04:05.508 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-02-20T13:04:05.570 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:05 +0000 (0:00:02.254) 0:02:03.217 ***** 2021-02-20T13:04:05.578 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-02-20T13:04:05.665 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:05 +0000 (0:00:00.096) 0:02:03.314 ***** 2021-02-20T13:04:06.604 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-02-20T13:04:06.606 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"cache_update_time": 1613825943, "cache_updated": false, "changed": false} 2021-02-20T13:04:06.650 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:06 +0000 (0:00:00.985) 0:02:04.300 ***** 2021-02-20T13:04:06.669 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"cache_update_time": 1613825899, "cache_updated": false, "changed": false} 2021-02-20T13:04:06.722 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:06 +0000 (0:00:00.071) 0:02:04.372 ***** 2021-02-20T13:04:06.732 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-02-20T13:04:06.775 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:06 +0000 (0:00:00.052) 0:02:04.425 ***** 2021-02-20T13:04:06.832 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:06 +0000 (0:00:00.056) 0:02:04.481 ***** 2021-02-20T13:04:06.971 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** [WARNING]: Consider using 'become', 'become_method', and 'become_user' rather than running sudo 2021-02-20T13:04:06.973 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.034343", "end": "2021-02-20 13:04:07.216767", "rc": 0, "start": "2021-02-20 13:04:07.182424", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:04:07.009 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:07 +0000 (0:00:00.177) 0:02:04.659 ***** 2021-02-20T13:04:07.050 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.033855", "end": "2021-02-20 13:04:06.817220", "rc": 0, "start": "2021-02-20 13:04:06.783365", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:04:07.098 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:07 +0000 (0:00:00.084) 0:02:04.743 ***** 2021-02-20T13:04:09.925 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2021-02-20T13:04:09.927 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"cache_update_time": 1613825899, "cache_updated": false, "changed": false} 2021-02-20T13:04:09.967 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:09 +0000 (0:00:02.870) 0:02:07.614 ***** 2021-02-20T13:04:10.074 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"cache_update_time": 1613825943, "cache_updated": false, "changed": false} 2021-02-20T13:04:10.111 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:10 +0000 (0:00:00.146) 0:02:07.760 ***** 2021-02-20T13:04:18.855 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2021-02-20T13:04:18.857 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 34 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137610 files and directorie 2021-02-20T13:04:18.858 INFO:teuthology.task.ansible.out:s currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 34 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Read 2021-02-20T13:04:18.860 INFO:teuthology.task.ansible.out:ing database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137610 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-02-20T13:04:18.897 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:18 +0000 (0:00:08.786) 0:02:16.547 ***** 2021-02-20T13:04:18.919 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 34 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137610 files and directorie 2021-02-20T13:04:18.921 INFO:teuthology.task.ansible.out:s currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 34 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Read 2021-02-20T13:04:18.923 INFO:teuthology.task.ansible.out:ing database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137610 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-02-20T13:04:18.967 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:18 +0000 (0:00:00.069) 0:02:16.616 ***** 2021-02-20T13:04:19.808 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-02-20T13:04:19.810 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"cache_update_time": 1613825899, "cache_updated": false, "changed": false} 2021-02-20T13:04:19.863 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:19 +0000 (0:00:00.895) 0:02:17.512 ***** 2021-02-20T13:04:19.901 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-02-20T13:04:19.904 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:19.943 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:19 +0000 (0:00:00.080) 0:02:17.592 ***** 2021-02-20T13:04:19.956 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-02-20T13:04:19.962 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"cache_update_time": 1613825943, "cache_updated": false, "changed": false} 2021-02-20T13:04:20.029 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:20 +0000 (0:00:00.085) 0:02:17.678 ***** 2021-02-20T13:04:20.072 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:20.110 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:20 +0000 (0:00:00.081) 0:02:17.760 ***** 2021-02-20T13:04:28.737 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* changed: [smithi170.front.sepia.ceph.com] => {"cache_update_time": 1613825899, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 34 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.35 [40.5 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.35 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.35 [5194 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.35 [868 kB]\nFetched 10.4 MB in 1s (9461 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database 2021-02-20T13:04:28.740 INFO:teuthology.task.ansible.out: ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137468 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...\r\nProcessing trigger 2021-02-20T13:04:28.741 INFO:teuthology.task.ansible.out:s for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 34 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.35 [40.5 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.35 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.35 [5194 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.35 [868 kB]", "Fetched 10.4 MB in 1s (9461 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Read 2021-02-20T13:04:28.743 INFO:teuthology.task.ansible.out:ing database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137468 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-02-20T13:04:28.795 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:28 +0000 (0:00:08.684) 0:02:26.445 ***** 2021-02-20T13:04:28.825 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"cache_update_time": 1613825943, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 34 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.35 [40.5 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.35 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.35 [5194 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.35 [868 kB]\nFetched 10.4 MB in 1s (9629 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database .. 2021-02-20T13:04:28.827 INFO:teuthology.task.ansible.out:. 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137468 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubunt 2021-02-20T13:04:28.828 INFO:teuthology.task.ansible.out:u1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 34 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.35 [40.5 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.35 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.35 [5194 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.35 [868 kB]", "Fetched 10.4 MB in 1s (9629 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(R 2021-02-20T13:04:28.830 INFO:teuthology.task.ansible.out:eading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137468 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-02-20T13:04:28.869 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:28 +0000 (0:00:00.073) 0:02:26.518 ***** 2021-02-20T13:04:29.744 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-02-20T13:04:29.746 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"cache_update_time": 1613825899, "cache_updated": false, "changed": false} 2021-02-20T13:04:29.784 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:29 +0000 (0:00:00.915) 0:02:27.434 ***** 2021-02-20T13:04:29.898 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"cache_update_time": 1613825943, "cache_updated": false, "changed": false} 2021-02-20T13:04:29.940 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:29 +0000 (0:00:00.154) 0:02:27.589 ***** 2021-02-20T13:04:29.965 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-02-20T13:04:29.966 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-02-20T13:04:30.017 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:30 +0000 (0:00:00.077) 0:02:27.667 ***** 2021-02-20T13:04:30.092 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2021-02-20T13:04:30.094 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:30.153 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:30 +0000 (0:00:00.135) 0:02:27.802 ***** 2021-02-20T13:04:30.164 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-02-20T13:04:30.173 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-02-20T13:04:30.277 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:30 +0000 (0:00:00.121) 0:02:27.924 ***** 2021-02-20T13:04:30.357 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2021-02-20T13:04:30.358 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:30.404 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:30 +0000 (0:00:00.129) 0:02:28.053 ***** 2021-02-20T13:04:30.412 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-02-20T13:04:30.413 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-02-20T13:04:30.452 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:30 +0000 (0:00:00.048) 0:02:28.102 ***** 2021-02-20T13:04:30.609 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-02-20T13:04:30.665 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:30 +0000 (0:00:00.212) 0:02:28.314 ***** 2021-02-20T13:04:30.982 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-02-20T13:04:30.985 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-02-20T13:04:31.033 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:31 +0000 (0:00:00.368) 0:02:28.683 ***** 2021-02-20T13:04:31.136 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-02-20T13:04:31.179 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:31 +0000 (0:00:00.145) 0:02:28.828 ***** 2021-02-20T13:04:31.207 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-02-20T13:04:31.210 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-02-20T13:04:31.265 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:31 +0000 (0:00:00.085) 0:02:28.914 ***** 2021-02-20T13:04:31.373 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-02-20T13:04:31.447 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:31 +0000 (0:00:00.182) 0:02:29.096 ***** 2021-02-20T13:04:31.451 INFO:teuthology.task.ansible.out: 2021-02-20T13:04:31.505 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2021-02-20T13:04:31.507 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T13:04:31.568 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:31 +0000 (0:00:00.113) 0:02:29.210 ***** 2021-02-20T13:04:31.793 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T13:04:31.835 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:31 +0000 (0:00:00.274) 0:02:29.484 ***** 2021-02-20T13:04:31.991 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-02-20T13:04:31.993 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sat 2021-02-20 13:01:31 UTC", "ActiveEnterTimestampMonotonic": "19229371", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "-.mount systemd-tmpfiles-setup.service systemd-journald.socket remote-fs.target sysinit.target system.slice nss-lookup.target basic.target network.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-02-20 13:01:31 UTC", "AssertTimestampMonotonic": "18909004", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-02-20 13:01:31 UTC", "ConditionTimestampMonotonic": "18909004", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2021-02-20T13:04:31.994 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1433", "ExecMainStartTimestamp": "Sat 2021-02-20 13:01:31 UTC", "ExecMainStartTimestampMonotonic": "19229358", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-02-20 13:01:31 UTC", "InactiveExitTimestampMonotonic": "18909665", "InvocationID": "a69876c9fbe2417dad2befb0a85aff2b", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2021-02-20T13:04:31.996 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1433", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2021-02-20T13:04:31.998 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 13:01:31 UTC", "StateChangeTimestampMonotonic": "19229371", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-02-20 13:01:31 UTC", "WatchdogTimestampMonotonic": "19229370", "WatchdogUSec": "0"}} 2021-02-20T13:04:32.044 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:32 +0000 (0:00:00.208) 0:02:29.693 ***** 2021-02-20T13:04:32.097 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:32.139 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:32 +0000 (0:00:00.093) 0:02:29.787 ***** 2021-02-20T13:04:32.162 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-02-20T13:04:32.166 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sat 2021-02-20 13:01:42 UTC", "ActiveEnterTimestampMonotonic": "22643623", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "system.slice -.mount sysinit.target basic.target systemd-journald.socket network.target systemd-tmpfiles-setup.service nss-lookup.target remote-fs.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-02-20 13:01:41 UTC", "AssertTimestampMonotonic": "22252618", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-02-20 13:01:41 UTC", "ConditionTimestampMonotonic": "22252618", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2021-02-20T13:04:32.168 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1359", "ExecMainStartTimestamp": "Sat 2021-02-20 13:01:42 UTC", "ExecMainStartTimestampMonotonic": "22643608", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-02-20 13:01:41 UTC", "InactiveExitTimestampMonotonic": "22253483", "InvocationID": "d036acf6b05f4f7cb7ccc90e9e54b674", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2021-02-20T13:04:32.170 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1359", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2021-02-20T13:04:32.171 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 13:01:42 UTC", "StateChangeTimestampMonotonic": "22643623", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-02-20 13:01:42 UTC", "WatchdogTimestampMonotonic": "22643621", "WatchdogUSec": "0"}} 2021-02-20T13:04:32.226 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:32 +0000 (0:00:00.088) 0:02:29.876 ***** 2021-02-20T13:04:32.234 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-02-20T13:04:32.236 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:32.273 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:32 +0000 (0:00:00.046) 0:02:29.922 ***** 2021-02-20T13:04:32.280 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-02-20T13:04:32.281 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:32.331 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:32 +0000 (0:00:00.058) 0:02:29.981 ***** 2021-02-20T13:04:32.356 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-02-20T13:04:32.358 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:32.396 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:32 +0000 (0:00:00.064) 0:02:30.046 ***** 2021-02-20T13:04:32.427 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:32.469 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:32 +0000 (0:00:00.071) 0:02:30.118 ***** 2021-02-20T13:04:32.479 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:32.524 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:32 +0000 (0:00:00.049) 0:02:30.167 ***** 2021-02-20T13:04:32.527 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-02-20T13:04:32.528 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:32.590 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:32 +0000 (0:00:00.072) 0:02:30.240 ***** 2021-02-20T13:04:32.631 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-02-20T13:04:32.633 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:32.689 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:32 +0000 (0:00:00.098) 0:02:30.338 ***** 2021-02-20T13:04:32.697 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-02-20T13:04:32.699 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:32.791 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:32 +0000 (0:00:00.102) 0:02:30.440 ***** 2021-02-20T13:04:32.801 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-02-20T13:04:32.803 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:32.857 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:32 +0000 (0:00:00.066) 0:02:30.507 ***** 2021-02-20T13:04:32.878 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:32.923 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:32 +0000 (0:00:00.065) 0:02:30.572 ***** 2021-02-20T13:04:32.933 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-02-20T13:04:32.934 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:33.001 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:32 +0000 (0:00:00.078) 0:02:30.650 ***** 2021-02-20T13:04:33.013 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-02-20T13:04:33.015 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:33.086 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:33 +0000 (0:00:00.077) 0:02:30.728 ***** TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:33.146 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:33 +0000 (0:00:00.068) 0:02:30.796 ***** 2021-02-20T13:04:33.159 INFO:teuthology.task.ansible.out: 2021-02-20T13:04:33.164 INFO:teuthology.task.ansible.out:TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-02-20T13:04:33.166 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:33.225 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:33 +0000 (0:00:00.078) 0:02:30.875 ***** 2021-02-20T13:04:33.246 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T13:04:33.247 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:33.292 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:33 +0000 (0:00:00.066) 0:02:30.941 ***** 2021-02-20T13:04:33.302 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-02-20T13:04:33.304 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:33.363 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:33 +0000 (0:00:00.071) 0:02:31.012 ***** 2021-02-20T13:04:33.371 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-02-20T13:04:33.373 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:33.437 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:33 +0000 (0:00:00.073) 0:02:31.086 ***** 2021-02-20T13:04:33.444 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T13:04:33.446 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:33.493 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:33 +0000 (0:00:00.056) 0:02:31.143 ***** 2021-02-20T13:04:33.525 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-02-20T13:04:33.527 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:33.581 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:33 +0000 (0:00:00.086) 0:02:31.229 ***** 2021-02-20T13:04:33.589 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-02-20T13:04:33.591 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:33.663 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:33 +0000 (0:00:00.083) 0:02:31.313 ***** 2021-02-20T13:04:33.703 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-02-20T13:04:33.704 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:33.746 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:33 +0000 (0:00:00.082) 0:02:31.396 ***** 2021-02-20T13:04:33.769 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-02-20T13:04:33.771 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:33.870 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:33 +0000 (0:00:00.123) 0:02:31.519 ***** 2021-02-20T13:04:33.883 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-02-20T13:04:33.885 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:33.950 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:33 +0000 (0:00:00.077) 0:02:31.596 ***** 2021-02-20T13:04:33.960 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-02-20T13:04:33.962 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:34.013 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:34 +0000 (0:00:00.066) 0:02:31.663 ***** 2021-02-20T13:04:34.032 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-20T13:04:34.033 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:34.101 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:34 +0000 (0:00:00.087) 0:02:31.751 ***** 2021-02-20T13:04:34.113 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:34.171 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:34 +0000 (0:00:00.070) 0:02:31.821 ***** 2021-02-20T13:04:34.178 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-02-20T13:04:34.180 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:34.227 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:34 +0000 (0:00:00.055) 0:02:31.877 ***** 2021-02-20T13:04:34.246 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-20T13:04:34.249 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:34.298 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:34 +0000 (0:00:00.070) 0:02:31.948 ***** 2021-02-20T13:04:34.310 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-02-20T13:04:34.312 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:34.363 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:34 +0000 (0:00:00.064) 0:02:32.013 ***** 2021-02-20T13:04:34.377 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-02-20T13:04:34.379 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:34.458 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:34 +0000 (0:00:00.094) 0:02:32.107 ***** 2021-02-20T13:04:34.471 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-02-20T13:04:34.472 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:34.541 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:34 +0000 (0:00:00.083) 0:02:32.191 ***** 2021-02-20T13:04:34.551 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-02-20T13:04:34.553 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:34.629 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:34 +0000 (0:00:00.088) 0:02:32.279 ***** 2021-02-20T13:04:34.640 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-02-20T13:04:34.642 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:34.686 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:34 +0000 (0:00:00.056) 0:02:32.335 ***** 2021-02-20T13:04:34.694 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-02-20T13:04:34.696 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:34.742 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:34 +0000 (0:00:00.053) 0:02:32.388 ***** 2021-02-20T13:04:34.750 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:34.829 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:34 +0000 (0:00:00.088) 0:02:32.476 ***** 2021-02-20T13:04:34.836 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-02-20T13:04:34.838 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:34.909 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:34 +0000 (0:00:00.077) 0:02:32.554 ***** 2021-02-20T13:04:34.917 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-20T13:04:34.917 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:34.977 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:34 +0000 (0:00:00.072) 0:02:32.627 ***** 2021-02-20T13:04:34.985 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-02-20T13:04:34.986 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:35.050 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:35 +0000 (0:00:00.072) 0:02:32.700 ***** 2021-02-20T13:04:35.092 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-02-20T13:04:35.094 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:35.163 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:35 +0000 (0:00:00.112) 0:02:32.812 ***** 2021-02-20T13:04:35.172 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:35.218 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:35 +0000 (0:00:00.054) 0:02:32.867 ***** 2021-02-20T13:04:35.220 INFO:teuthology.task.ansible.out: 2021-02-20T13:04:35.229 INFO:teuthology.task.ansible.out: 2021-02-20T13:04:35.231 INFO:teuthology.task.ansible.out:TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:35.286 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:35 +0000 (0:00:00.068) 0:02:32.935 ***** 2021-02-20T13:04:35.307 INFO:teuthology.task.ansible.out: 2021-02-20T13:04:35.310 INFO:teuthology.task.ansible.out:TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-02-20T13:04:35.312 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:35.362 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:35 +0000 (0:00:00.076) 0:02:33.011 ***** 2021-02-20T13:04:35.363 INFO:teuthology.task.ansible.out: 2021-02-20T13:04:35.375 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-20T13:04:35.376 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:35.378 INFO:teuthology.task.ansible.out: 2021-02-20T13:04:35.552 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:35 +0000 (0:00:00.190) 0:02:33.202 ***** 2021-02-20T13:04:35.587 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-02-20T13:04:35.588 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:35.660 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:35 +0000 (0:00:00.108) 0:02:33.310 ***** 2021-02-20T13:04:35.701 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-02-20T13:04:35.703 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:35.759 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:35 +0000 (0:00:00.097) 0:02:33.407 ***** 2021-02-20T13:04:35.787 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-20T13:04:35.789 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:35.864 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:35 +0000 (0:00:00.105) 0:02:33.513 ***** 2021-02-20T13:04:35.883 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:35.993 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:35 +0000 (0:00:00.128) 0:02:33.642 ***** 2021-02-20T13:04:36.071 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-02-20T13:04:36.073 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:36.139 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:36 +0000 (0:00:00.145) 0:02:33.788 ***** 2021-02-20T13:04:36.146 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:36.192 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:36 +0000 (0:00:00.053) 0:02:33.841 ***** 2021-02-20T13:04:36.214 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-02-20T13:04:36.216 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:36.263 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:36 +0000 (0:00:00.071) 0:02:33.912 ***** 2021-02-20T13:04:36.312 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2021-02-20T13:04:36.314 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:36.370 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:36 +0000 (0:00:00.106) 0:02:34.019 ***** 2021-02-20T13:04:36.474 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-02-20T13:04:36.476 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-02-20T13:04:36.541 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:36 +0000 (0:00:00.170) 0:02:34.190 ***** 2021-02-20T13:04:36.589 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-02-20T13:04:36.636 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:36 +0000 (0:00:00.095) 0:02:34.285 ***** 2021-02-20T13:04:37.153 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-02-20T13:04:37.155 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-02-20T13:04:37.194 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:37 +0000 (0:00:00.558) 0:02:34.843 ***** 2021-02-20T13:04:37.199 INFO:teuthology.task.ansible.out: 2021-02-20T13:04:37.216 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-02-20T13:04:37.265 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:37 +0000 (0:00:00.070) 0:02:34.914 ***** 2021-02-20T13:04:37.276 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-02-20T13:04:37.278 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:37.359 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:37 +0000 (0:00:00.094) 0:02:35.009 ***** 2021-02-20T13:04:37.367 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:37.413 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:37 +0000 (0:00:00.053) 0:02:35.063 ***** 2021-02-20T13:04:37.877 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-02-20T13:04:37.878 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-02-20T13:04:37.915 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:37 +0000 (0:00:00.502) 0:02:35.565 ***** 2021-02-20T13:04:37.937 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-02-20T13:04:38.008 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:38 +0000 (0:00:00.092) 0:02:35.658 ***** 2021-02-20T13:04:38.547 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-02-20T13:04:38.551 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-02-20T13:04:38.615 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:38 +0000 (0:00:00.606) 0:02:36.265 ***** 2021-02-20T13:04:38.633 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-02-20T13:04:38.700 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:38 +0000 (0:00:00.084) 0:02:36.349 ***** 2021-02-20T13:04:39.101 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-02-20T13:04:39.143 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:39 +0000 (0:00:00.443) 0:02:36.793 ***** 2021-02-20T13:04:39.156 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-02-20T13:04:39.232 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:39 +0000 (0:00:00.089) 0:02:36.882 ***** 2021-02-20T13:04:39.374 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-02-20T13:04:39.376 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T13:04:39.446 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:39 +0000 (0:00:00.213) 0:02:37.095 ***** 2021-02-20T13:04:39.490 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T13:04:39.539 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:39 +0000 (0:00:00.092) 0:02:37.188 ***** 2021-02-20T13:04:39.545 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-02-20T13:04:39.547 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:39.592 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:39 +0000 (0:00:00.053) 0:02:37.242 ***** 2021-02-20T13:04:39.600 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:39.664 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:39 +0000 (0:00:00.071) 0:02:37.313 ***** 2021-02-20T13:04:39.698 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-02-20T13:04:39.700 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:39.746 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:39 +0000 (0:00:00.080) 0:02:37.393 ***** 2021-02-20T13:04:39.753 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:39.829 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:39 +0000 (0:00:00.084) 0:02:37.478 ***** 2021-02-20T13:04:39.838 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-02-20T13:04:39.840 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:39.893 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:39 +0000 (0:00:00.064) 0:02:37.542 ***** 2021-02-20T13:04:39.901 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:39.989 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:39 +0000 (0:00:00.095) 0:02:37.638 ***** 2021-02-20T13:04:40.017 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2021-02-20T13:04:40.019 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:40.064 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:40 +0000 (0:00:00.075) 0:02:37.714 ***** 2021-02-20T13:04:40.083 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:40.126 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:40 +0000 (0:00:00.061) 0:02:37.775 ***** 2021-02-20T13:04:40.138 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-02-20T13:04:40.140 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:40.201 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:40 +0000 (0:00:00.075) 0:02:37.850 ***** 2021-02-20T13:04:40.209 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:40.253 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:40 +0000 (0:00:00.051) 0:02:37.902 ***** 2021-02-20T13:04:40.254 INFO:teuthology.task.ansible.out: 2021-02-20T13:04:40.265 INFO:teuthology.task.ansible.out: 2021-02-20T13:04:40.267 INFO:teuthology.task.ansible.out:TASK [testnode : set_fact] ***************************************************** ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-02-20T13:04:40.308 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:40 +0000 (0:00:00.054) 0:02:37.957 ***** 2021-02-20T13:04:40.309 INFO:teuthology.task.ansible.out: 2021-02-20T13:04:40.329 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-02-20T13:04:40.367 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:40 +0000 (0:00:00.059) 0:02:38.017 ***** 2021-02-20T13:04:40.424 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi170.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-02-20T13:04:40.445 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-02-20T13:04:40.498 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2021-02-20T13:04:40.509 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2021-02-20T13:04:40.521 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-02-20T13:04:40.533 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2021-02-20T13:04:40.544 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} ok: [smithi170.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-02-20T13:04:40.555 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-02-20T13:04:40.567 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-02-20T13:04:40.575 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-02-20T13:04:40.577 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} skipping: [smithi170.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-02-20T13:04:40.588 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-02-20T13:04:40.599 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-02-20T13:04:40.613 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-02-20T13:04:40.702 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:40 +0000 (0:00:00.334) 0:02:38.351 ***** 2021-02-20T13:04:40.710 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-02-20T13:04:40.712 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-02-20T13:04:40.713 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2021-02-20T13:04:40.713 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-02-20T13:04:40.768 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:40 +0000 (0:00:00.066) 0:02:38.418 ***** 2021-02-20T13:04:41.219 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi170.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.031241", "end": "2021-02-20 13:04:41.166835", "rc": 0, "start": "2021-02-20 13:04:41.135594", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/virtualenv/bin/teuthology-lock"]} changed: [smithi183.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.049650", "end": "2021-02-20 13:04:41.175780", "rc": 0, "start": "2021-02-20 13:04:41.126130", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/virtualenv/bin/teuthology-lock"]} 2021-02-20T13:04:41.262 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:41 +0000 (0:00:00.494) 0:02:38.912 ***** 2021-02-20T13:04:41.345 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:41 +0000 (0:00:00.082) 0:02:38.994 ***** 2021-02-20T13:04:41.365 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2021-02-20T13:04:41.367 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:41.421 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:41 +0000 (0:00:00.076) 0:02:39.071 ***** 2021-02-20T13:04:41.445 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:41.521 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:41 +0000 (0:00:00.099) 0:02:39.171 ***** 2021-02-20T13:04:41.531 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-02-20T13:04:41.533 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:41.593 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:41 +0000 (0:00:00.072) 0:02:39.243 ***** 2021-02-20T13:04:41.608 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:41.668 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:41 +0000 (0:00:00.071) 0:02:39.314 ***** 2021-02-20T13:04:41.691 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-02-20T13:04:41.693 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:41.778 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:41 +0000 (0:00:00.113) 0:02:39.428 ***** 2021-02-20T13:04:41.799 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:41.886 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:41 +0000 (0:00:00.107) 0:02:39.535 ***** 2021-02-20T13:04:42.782 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-02-20T13:04:42.784 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"cache_update_time": 1613825899, "cache_updated": false, "changed": false} 2021-02-20T13:04:42.833 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:42 +0000 (0:00:00.947) 0:02:40.482 ***** 2021-02-20T13:04:42.884 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-02-20T13:04:42.886 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:42.941 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:42 +0000 (0:00:00.108) 0:02:40.591 ***** 2021-02-20T13:04:42.960 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-02-20T13:04:42.962 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"cache_update_time": 1613825943, "cache_updated": false, "changed": false} 2021-02-20T13:04:42.999 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:42 +0000 (0:00:00.057) 0:02:40.648 ***** 2021-02-20T13:04:43.009 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-02-20T13:04:43.010 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'block_used': 13754883, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226340225, 'size_available': 927089561600, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226340225, "block_size": 4096, "block_total": 240095108, "block_used": 13754883, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927089561600, "size_total": 983429562368, "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}} 2021-02-20T13:04:43.053 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:43 +0000 (0:00:00.053) 0:02:40.702 ***** 2021-02-20T13:04:43.061 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:43.107 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:43 +0000 (0:00:00.053) 0:02:40.756 ***** 2021-02-20T13:04:43.168 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-02-20T13:04:43.170 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item={'block_used': 13754883, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226340225, 'size_available': 927089561600, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226340225, "block_size": 4096, "block_total": 240095108, "block_used": 13754883, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927089561600, "size_total": 983429562368, "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}} 2021-02-20T13:04:43.208 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:43 +0000 (0:00:00.101) 0:02:40.857 ***** 2021-02-20T13:04:43.244 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-02-20T13:04:43.246 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.010924", "end": "2021-02-20 13:04:43.029319", "rc": 0, "start": "2021-02-20 13:04:43.018395", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:04:43.311 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:43 +0000 (0:00:00.103) 0:02:40.960 ***** 2021-02-20T13:04:43.367 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.007427", "end": "2021-02-20 13:04:43.587812", "rc": 0, "start": "2021-02-20 13:04:43.580385", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:04:43.413 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:43 +0000 (0:00:00.101) 0:02:41.062 ***** 2021-02-20T13:04:43.430 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-02-20T13:04:43.432 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'block_used': 13754883, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226340225, 'size_available': 927089561600, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226340225, "block_size": 4096, "block_total": 240095108, "block_used": 13754883, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927089561600, "size_total": 983429562368, "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:43.480 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:43 +0000 (0:00:00.066) 0:02:41.129 ***** 2021-02-20T13:04:43.497 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'block_used': 13754883, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226340225, 'size_available': 927089561600, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226340225, "block_size": 4096, "block_total": 240095108, "block_used": 13754883, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927089561600, "size_total": 983429562368, "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:43.547 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:43 +0000 (0:00:00.067) 0:02:41.196 ***** 2021-02-20T13:04:43.555 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-02-20T13:04:43.606 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:43 +0000 (0:00:00.058) 0:02:41.255 ***** 2021-02-20T13:04:43.679 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:43 +0000 (0:00:00.073) 0:02:41.329 ***** 2021-02-20T13:04:43.703 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** skipping: [smithi170.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:43.761 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:43.762 INFO:teuthology.task.ansible.out: 2021-02-20T13:04:45.514 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-ERZzyVMqkyav4jvcEIGsO3uBWIBLLKStjVzHPVay4qLbEPEQCpTH3Q2LQoC57w9z'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.703607", "end": "2021-02-20 13:04:45.308095", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-ERZzyVMqkyav4jvcEIGsO3uBWIBLLKStjVzHPVay4qLbEPEQCpTH3Q2LQoC57w9z"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:43.604488", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T13:04:45.536 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:45.548 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-k5uk9Bo5un2nqQhYh1mMLGtpg7Jz0zaqVErxNEbTFb4FUNdK71qddW69cged46h2'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.640443", "end": "2021-02-20 13:04:45.774820", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-k5uk9Bo5un2nqQhYh1mMLGtpg7Jz0zaqVErxNEbTFb4FUNdK71qddW69cged46h2"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:44.134377", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T13:04:45.550 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:45.572 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:45.594 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:46.821 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-ERZzyVMqkyav4jvcEIGsO3uBWIBLLKStOQP4hiRavU48j01VKNG6dDmhtTm9UAv5'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.116294", "end": "2021-02-20 13:04:46.606099", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-ERZzyVMqkyav4jvcEIGsO3uBWIBLLKStOQP4hiRavU48j01VKNG6dDmhtTm9UAv5"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:45.489805", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T13:04:46.833 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XK1Y', 'wwn-0x5000c50092940bdb'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XK1Y-part1', 'wwn-0x5000c50092940bdb-part1'], 'uuids': ['602a9f0b-ccf6-4d5d-877a-542e0952c07a']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50092940bdb', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XK1Y', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XK1Y", "wwn-0x5000c50092940bdb"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XK1Y-part1", "wwn-0x5000c50092940bdb-part1"], "labels": [], "masters": [], "uuids": ["602a9f0b-ccf6-4d5d-877a-542e0952c07a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XK1Y", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50092940bdb"}}, "skip_reason": 2021-02-20T13:04:46.836 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2021-02-20T13:04:46.910 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-k5uk9Bo5un2nqQhYh1mMLGtpg7Jz0zaqa9zp1bAeJ0QehRUTlaeksZfFXCLPpYOV'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.140378", "end": "2021-02-20 13:04:47.125623", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-k5uk9Bo5un2nqQhYh1mMLGtpg7Jz0zaqa9zp1bAeJ0QehRUTlaeksZfFXCLPpYOV"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:45.985245", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T13:04:46.923 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XFFN', 'wwn-0x5000c50092929219'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XFFN-part1', 'wwn-0x5000c50092929219-part1'], 'uuids': ['602a9f0b-ccf6-4d5d-877a-542e0952c07a']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50092929219', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XFFN', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XFFN", "wwn-0x5000c50092929219"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XFFN-part1", "wwn-0x5000c50092929219-part1"], "labels": [], "masters": [], "uuids": ["602a9f0b-ccf6-4d5d-877a-542e0952c07a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XFFN", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50092929219"}}, "skip_reason": 2021-02-20T13:04:46.927 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2021-02-20T13:04:48.199 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '937703088', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-5KCP5i-zomT-FR63-B3rU-NJii-WQke-zkaNgq', 'nvme-INTEL_SSDPED1D480GA_PHMB7476002R480DGN', 'nvme-nvme.8086-50484d42373437363030325234383044474e-494e54454c2053534450454431443438304741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPED1D480GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '447.13 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.190132", "end": "2021-02-20 13:04:47.984232", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-5KCP5i-zomT-FR63-B3rU-NJii-WQke-zkaNgq", "nvme-INTEL_SSDPED1D480GA_PHMB7476002R480DGN", "nvme-nvme.8086-50484d42373437363030325234383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "937703088", "sectorsize": "512", "size": "447.13 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:46.794100", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now pa 2021-02-20T13:04:48.202 INFO:teuthology.task.ansible.out:rtition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T13:04:48.233 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:48.267 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:48.280 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-lvzGoG-FdkJ-4WBI-cdRn-7zeq-0M0U-Qf3n6G', 'nvme-INTEL_SSDPEDMD400G4_PHFT620500QJ400BGN', 'nvme-nvme.8086-50484654363230353030514a34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.210104", "end": "2021-02-20 13:04:48.488262", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-lvzGoG-FdkJ-4WBI-cdRn-7zeq-0M0U-Qf3n6G", "nvme-INTEL_SSDPEDMD400G4_PHFT620500QJ400BGN", "nvme-nvme.8086-50484654363230353030514a34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:47.278158", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2021-02-20T13:04:48.282 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T13:04:48.291 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:48.302 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:49.654 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-ERZzyVMqkyav4jvcEIGsO3uBWIBLLKSt8evDtP46F2ZiaNLGyW7k3xN8aIfZkgYI'], 'uuids': ['f4e23592-456d-4709-8167-9236337d8d0d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.133817", "end": "2021-02-20 13:04:49.440576", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-ERZzyVMqkyav4jvcEIGsO3uBWIBLLKSt8evDtP46F2ZiaNLGyW7k3xN8aIfZkgYI"], "labels": [], "masters": [], "uuids": ["f4e23592-456d-4709-8167-9236337d8d0d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:48.306759", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T13:04:49.666 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-k5uk9Bo5un2nqQhYh1mMLGtpg7Jz0zaqLQeuMv5cZ5YSNCXHEbi105i9v7Wsgmgq'], 'uuids': ['86754257-9c78-45e9-b121-d09a961e12da']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.142885", "end": "2021-02-20 13:04:49.880816", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-k5uk9Bo5un2nqQhYh1mMLGtpg7Jz0zaqLQeuMv5cZ5YSNCXHEbi105i9v7Wsgmgq"], "labels": [], "masters": [], "uuids": ["86754257-9c78-45e9-b121-d09a961e12da"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:48.737931", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T13:04:49.667 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:49.669 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:49.680 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:49.681 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:50.956 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-ERZzyVMqkyav4jvcEIGsO3uBWIBLLKSt0pt3u6Dd4QBo9WsctxwIqE6D0Wa0OjWo'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.129988", "end": "2021-02-20 13:04:50.733502", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-ERZzyVMqkyav4jvcEIGsO3uBWIBLLKSt0pt3u6Dd4QBo9WsctxwIqE6D0Wa0OjWo"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:49.603514", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T13:04:50.970 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-k5uk9Bo5un2nqQhYh1mMLGtpg7Jz0zaq4ahW6yTChGmtmq1u7U8jbI8k3v1sI8rf'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.144751", "end": "2021-02-20 13:04:51.181930", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-k5uk9Bo5un2nqQhYh1mMLGtpg7Jz0zaq4ahW6yTChGmtmq1u7U8jbI8k3v1sI8rf"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:50.037179", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T13:04:50.980 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:50.981 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:52.258 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-k5uk9Bo5un2nqQhYh1mMLGtpg7Jz0zaqM03Gq1SKTyP1742NWKkctQVJz3uMRfbM'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.127018", "end": "2021-02-20 13:04:52.473122", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-k5uk9Bo5un2nqQhYh1mMLGtpg7Jz0zaqM03Gq1SKTyP1742NWKkctQVJz3uMRfbM"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:51.346104", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T13:04:52.354 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:52 +0000 (0:00:08.674) 0:02:50.003 ***** 2021-02-20T13:04:52.385 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-ERZzyVMqkyav4jvcEIGsO3uBWIBLLKStCuM31oIce67f0uiQ7YM1ce1Z0zObbtnx'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.158533", "end": "2021-02-20 13:04:52.107619", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-ERZzyVMqkyav4jvcEIGsO3uBWIBLLKStCuM31oIce67f0uiQ7YM1ce1Z0zObbtnx"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:50.949086", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T13:04:52.466 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:52 +0000 (0:00:00.111) 0:02:50.115 ***** 2021-02-20T13:04:52.474 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-02-20T13:04:52.476 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:52.552 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:52.802 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-k5uk9Bo5un2nqQhYh1mMLGtpg7Jz0zaqVErxNEbTFb4FUNdK71qddW69cged46h2'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.153871", "end": "2021-02-20 13:04:53.012020", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-k5uk9Bo5un2nqQhYh1mMLGtpg7Jz0zaqVErxNEbTFb4FUNdK71qddW69cged46h2"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:52.858149", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.149282 s, 773 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.149282 s, 773 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T13:04:52.816 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-ERZzyVMqkyav4jvcEIGsO3uBWIBLLKStjVzHPVay4qLbEPEQCpTH3Q2LQoC57w9z'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.114823", "end": "2021-02-20 13:04:52.602516", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-ERZzyVMqkyav4jvcEIGsO3uBWIBLLKStjVzHPVay4qLbEPEQCpTH3Q2LQoC57w9z"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "1874903 2021-02-20T13:04:52.820 INFO:teuthology.task.ansible.out:04", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:52.487693", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.110132 s, 1.0 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.110132 s, 1.0 GB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T13:04:52.835 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:52.848 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:52.911 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:53.153 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-k5uk9Bo5un2nqQhYh1mMLGtpg7Jz0zaqa9zp1bAeJ0QehRUTlaeksZfFXCLPpYOV'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.151085", "end": "2021-02-20 13:04:53.363344", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-k5uk9Bo5un2nqQhYh1mMLGtpg7Jz0zaqa9zp1bAeJ0QehRUTlaeksZfFXCLPpYOV"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:53.212259", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.148312 s, 778 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.148312 s, 778 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T13:04:53.165 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XFFN', 'wwn-0x5000c50092929219'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XFFN-part1', 'wwn-0x5000c50092929219-part1'], 'uuids': ['602a9f0b-ccf6-4d5d-877a-542e0952c07a']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50092929219', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XFFN', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XFFN", "wwn-0x5000c50092929219"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XFFN-part1", "wwn-0x5000c50092929219-part1"], "labels": [], "masters": [], "uuids": ["602a9f0b-ccf6-4d5d-877a-542e0952c07a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XFFN", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50092929219"}}, "skip_reason": 2021-02-20T13:04:53.167 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2021-02-20T13:04:53.220 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-ERZzyVMqkyav4jvcEIGsO3uBWIBLLKStOQP4hiRavU48j01VKNG6dDmhtTm9UAv5'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.111606", "end": "2021-02-20 13:04:52.998794", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-ERZzyVMqkyav4jvcEIGsO3uBWIBLLKStOQP4hiRavU48j01VKNG6dDmhtTm9UAv5"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:52.887188", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.106789 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.106789 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T13:04:53.242 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XK1Y', 'wwn-0x5000c50092940bdb'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XK1Y-part1', 'wwn-0x5000c50092940bdb-part1'], 'uuids': ['602a9f0b-ccf6-4d5d-877a-542e0952c07a']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50092940bdb', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XK1Y', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XK1Y", "wwn-0x5000c50092940bdb"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XK1Y-part1", "wwn-0x5000c50092940bdb-part1"], "labels": [], "masters": [], "uuids": ["602a9f0b-ccf6-4d5d-877a-542e0952c07a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XK1Y", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50092940bdb"}}, "skip_reason": 2021-02-20T13:04:53.244 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2021-02-20T13:04:53.436 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-lvzGoG-FdkJ-4WBI-cdRn-7zeq-0M0U-Qf3n6G', 'nvme-INTEL_SSDPEDMD400G4_PHFT620500QJ400BGN', 'nvme-nvme.8086-50484654363230353030514a34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.050427", "end": "2021-02-20 13:04:53.659204", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-lvzGoG-FdkJ-4WBI-cdRn-7zeq-0M0U-Qf3n6G", "nvme-INTEL_SSDPEDMD400G4_PHFT620500QJ400BGN", "nvme-nvme.8086-50484654363230353030514a34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:53.608777", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0458652 s, 2.5 GB/s", "std 2021-02-20T13:04:53.438 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0458652 s, 2.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T13:04:53.448 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:53.466 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '937703088', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-5KCP5i-zomT-FR63-B3rU-NJii-WQke-zkaNgq', 'nvme-INTEL_SSDPED1D480GA_PHMB7476002R480DGN', 'nvme-nvme.8086-50484d42373437363030325234383044474e-494e54454c2053534450454431443438304741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPED1D480GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '447.13 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.054641", "end": "2021-02-20 13:04:53.255952", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-5KCP5i-zomT-FR63-B3rU-NJii-WQke-zkaNgq", "nvme-INTEL_SSDPED1D480GA_PHMB7476002R480DGN", "nvme-nvme.8086-50484d42373437363030325234383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "937703088", "sectorsize": "512", "size": "447.13 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:53.201311", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0498759 s, 2.3 GB/s", "stderr_lines": [" 2021-02-20T13:04:53.468 INFO:teuthology.task.ansible.out:110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0498759 s, 2.3 GB/s"], "stdout": "", "stdout_lines": []} skipping: [smithi183.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:53.479 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:53.491 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:53.747 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-ERZzyVMqkyav4jvcEIGsO3uBWIBLLKSt8evDtP46F2ZiaNLGyW7k3xN8aIfZkgYI'], 'uuids': ['f4e23592-456d-4709-8167-9236337d8d0d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.110614", "end": "2021-02-20 13:04:53.539979", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-ERZzyVMqkyav4jvcEIGsO3uBWIBLLKSt8evDtP46F2ZiaNLGyW7k3xN8aIfZkgYI"], "labels": [], "masters": [], "uuids": ["f4e23592-456d-4709-8167-9236337d8d0d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:53.429365", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.107241 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.107241 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T13:04:53.751 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-k5uk9Bo5un2nqQhYh1mMLGtpg7Jz0zaqLQeuMv5cZ5YSNCXHEbi105i9v7Wsgmgq'], 'uuids': ['86754257-9c78-45e9-b121-d09a961e12da']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.136745", "end": "2021-02-20 13:04:53.967698", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-k5uk9Bo5un2nqQhYh1mMLGtpg7Jz0zaqLQeuMv5cZ5YSNCXHEbi105i9v7Wsgmgq"], "labels": [], "masters": [], "uuids": ["86754257-9c78-45e9-b121-d09a961e12da"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:53.830953", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.132821 s, 868 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.132821 s, 868 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T13:04:53.761 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:53.773 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:53.784 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:53.806 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:54.078 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-ERZzyVMqkyav4jvcEIGsO3uBWIBLLKSt0pt3u6Dd4QBo9WsctxwIqE6D0Wa0OjWo'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.110842", "end": "2021-02-20 13:04:53.869819", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-ERZzyVMqkyav4jvcEIGsO3uBWIBLLKSt0pt3u6Dd4QBo9WsctxwIqE6D0Wa0OjWo"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:53.758977", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.106115 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.106115 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T13:04:54.095 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:54.107 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-k5uk9Bo5un2nqQhYh1mMLGtpg7Jz0zaq4ahW6yTChGmtmq1u7U8jbI8k3v1sI8rf'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.161222", "end": "2021-02-20 13:04:54.318563", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-k5uk9Bo5un2nqQhYh1mMLGtpg7Jz0zaq4ahW6yTChGmtmq1u7U8jbI8k3v1sI8rf"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:54.157341", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.156685 s, 736 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.156685 s, 736 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T13:04:54.119 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:54.361 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-ERZzyVMqkyav4jvcEIGsO3uBWIBLLKStCuM31oIce67f0uiQ7YM1ce1Z0zObbtnx'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.112310", "end": "2021-02-20 13:04:54.154323", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-ERZzyVMqkyav4jvcEIGsO3uBWIBLLKStCuM31oIce67f0uiQ7YM1ce1Z0zObbtnx"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:54.042013", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.10796 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.10796 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T13:04:54.447 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:54 +0000 (0:00:01.981) 0:02:52.097 ***** 2021-02-20T13:04:54.461 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-k5uk9Bo5un2nqQhYh1mMLGtpg7Jz0zaqM03Gq1SKTyP1742NWKkctQVJz3uMRfbM'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.158932", "end": "2021-02-20 13:04:54.661723", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-k5uk9Bo5un2nqQhYh1mMLGtpg7Jz0zaqM03Gq1SKTyP1742NWKkctQVJz3uMRfbM"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 13:04:54.502791", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.154424 s, 747 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.154424 s, 747 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T13:04:54.533 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:54 +0000 (0:00:00.085) 0:02:52.182 ***** 2021-02-20T13:04:54.759 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-02-20T13:04:54.760 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.143075", "end": "2021-02-20 13:04:54.544243", "rc": 0, "start": "2021-02-20 13:04:54.401168", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:04:54.805 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:54 +0000 (0:00:00.271) 0:02:52.454 ***** 2021-02-20T13:04:54.845 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.138779", "end": "2021-02-20 13:04:55.044077", "rc": 0, "start": "2021-02-20 13:04:54.905298", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:04:54.936 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:54 +0000 (0:00:00.130) 0:02:52.585 ***** 2021-02-20T13:04:55.084 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-02-20T13:04:55.086 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.043746", "end": "2021-02-20 13:04:55.050041", "rc": 0, "start": "2021-02-20 13:04:55.006295", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:04:55.132 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:55 +0000 (0:00:00.194) 0:02:52.780 ***** 2021-02-20T13:04:55.171 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.068636", "end": "2021-02-20 13:04:55.374404", "rc": 0, "start": "2021-02-20 13:04:55.305768", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:04:55.173 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-02-20T13:04:55.217 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:55 +0000 (0:00:00.086) 0:02:52.867 ***** 2021-02-20T13:04:55.269 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:55 +0000 (0:00:00.051) 0:02:52.918 ***** 2021-02-20T13:04:55.278 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-02-20T13:04:55.280 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:55.339 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:55 +0000 (0:00:00.064) 0:02:52.982 ***** 2021-02-20T13:04:55.345 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-02-20T13:04:55.398 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:55 +0000 (0:00:00.065) 0:02:53.048 ***** 2021-02-20T13:04:55.405 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-02-20T13:04:55.407 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:55.475 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:55 +0000 (0:00:00.076) 0:02:53.124 ***** 2021-02-20T13:04:55.487 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-02-20T13:04:55.488 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:55.533 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:55 +0000 (0:00:00.058) 0:02:53.183 ***** 2021-02-20T13:04:55.542 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-02-20T13:04:55.544 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:55.597 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:55 +0000 (0:00:00.061) 0:02:53.245 ***** 2021-02-20T13:04:55.614 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-02-20T13:04:55.619 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:55.660 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:55 +0000 (0:00:00.064) 0:02:53.309 ***** 2021-02-20T13:04:55.678 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-02-20T13:04:55.680 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:55.739 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:55 +0000 (0:00:00.079) 0:02:53.388 ***** 2021-02-20T13:04:55.748 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-02-20T13:04:55.749 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:55.793 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:55 +0000 (0:00:00.053) 0:02:53.442 ***** 2021-02-20T13:04:55.831 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:55.878 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:55 +0000 (0:00:00.084) 0:02:53.527 ***** 2021-02-20T13:04:55.917 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ ok: [smithi170.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T13:04:55.978 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:55 +0000 (0:00:00.100) 0:02:53.628 ***** 2021-02-20T13:04:56.048 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-02-20T13:04:56.050 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'block_used': 13754883, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226340225, 'size_available': 927089561600, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226340225, "block_size": 4096, "block_total": 240095108, "block_used": 13754883, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927089561600, "size_total": 983429562368, "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:56.096 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:56 +0000 (0:00:00.117) 0:02:53.746 ***** 2021-02-20T13:04:56.110 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-02-20T13:04:56.112 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T13:04:56.158 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:56 +0000 (0:00:00.061) 0:02:53.808 ***** 2021-02-20T13:04:56.177 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-02-20T13:04:56.179 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:56.237 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:56 +0000 (0:00:00.078) 0:02:53.886 ***** 2021-02-20T13:04:56.247 INFO:teuthology.task.ansible.out: 2021-02-20T13:04:56.249 INFO:teuthology.task.ansible.out:TASK [testnode : Set root disk] ************************************************ skipping: [smithi183.front.sepia.ceph.com] => (item={'block_used': 13754883, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226340225, 'size_available': 927089561600, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226340225, "block_size": 4096, "block_total": 240095108, "block_used": 13754883, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927089561600, "size_total": 983429562368, "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:56.294 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:56 +0000 (0:00:00.057) 0:02:53.944 ***** 2021-02-20T13:04:56.304 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T13:04:56.305 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:56.349 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:56 +0000 (0:00:00.052) 0:02:53.996 ***** 2021-02-20T13:04:56.355 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-02-20T13:04:56.357 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:56.411 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:56 +0000 (0:00:00.064) 0:02:54.061 ***** 2021-02-20T13:04:56.418 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T13:04:56.420 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:56.470 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:56 +0000 (0:00:00.058) 0:02:54.120 ***** 2021-02-20T13:04:56.480 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T13:04:56.482 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:56.521 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:56 +0000 (0:00:00.050) 0:02:54.170 ***** 2021-02-20T13:04:56.528 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-02-20T13:04:56.530 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:56.572 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:56 +0000 (0:00:00.051) 0:02:54.222 ***** 2021-02-20T13:04:56.583 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:56.623 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:56 +0000 (0:00:00.050) 0:02:54.273 ***** 2021-02-20T13:04:56.634 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:56.684 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:56 +0000 (0:00:00.060) 0:02:54.333 ***** 2021-02-20T13:04:56.699 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-02-20T13:04:56.701 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:56.752 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:56 +0000 (0:00:00.068) 0:02:54.402 ***** 2021-02-20T13:04:56.760 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-02-20T13:04:56.763 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:56.807 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:56 +0000 (0:00:00.054) 0:02:54.456 ***** 2021-02-20T13:04:56.816 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-02-20T13:04:56.818 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:56.866 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:56 +0000 (0:00:00.059) 0:02:54.516 ***** 2021-02-20T13:04:56.907 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:56.952 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:56 +0000 (0:00:00.086) 0:02:54.602 ***** 2021-02-20T13:04:57.495 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-02-20T13:04:57.496 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-02-20T13:04:57.537 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:57 +0000 (0:00:00.584) 0:02:55.186 ***** 2021-02-20T13:04:57.582 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:57 +0000 (0:00:00.045) 0:02:55.231 ***** 2021-02-20T13:04:58.099 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-02-20T13:04:58.101 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T13:04:58.153 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T13:04:58.376 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T13:04:58.388 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T13:04:58.591 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T13:04:58.593 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T13:04:58.814 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T13:04:58.848 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T13:04:58.850 INFO:teuthology.task.ansible.out: 2021-02-20T13:04:59.024 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T13:04:59.067 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:59 +0000 (0:00:01.484) 0:02:56.716 ***** 2021-02-20T13:04:59.106 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T13:04:59.145 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:59 +0000 (0:00:00.078) 0:02:56.794 ***** 2021-02-20T13:04:59.219 INFO:teuthology.task.ansible.out: 2021-02-20T13:04:59.221 INFO:teuthology.task.ansible.out:TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** changed: [smithi183.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-02-20T13:04:59.282 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:59 +0000 (0:00:00.131) 0:02:56.926 ***** 2021-02-20T13:04:59.305 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-02-20T13:04:59.351 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:59 +0000 (0:00:00.074) 0:02:57.000 ***** 2021-02-20T13:04:59.443 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-02-20T13:04:59.445 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T13:04:59.519 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T13:04:59.578 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T13:04:59.643 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T13:04:59.706 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T13:04:59.760 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T13:04:59.826 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T13:04:59.859 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:59.902 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:59 +0000 (0:00:00.551) 0:02:57.552 ***** 2021-02-20T13:04:59.910 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T13:04:59.912 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-02-20T13:04:59.954 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:59 +0000 (0:00:00.051) 0:02:57.604 ***** 2021-02-20T13:04:59.961 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-02-20T13:04:59.963 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:00.002 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:04:59 +0000 (0:00:00.046) 0:02:57.650 ***** 2021-02-20T13:05:00.009 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:00.055 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:00 +0000 (0:00:00.054) 0:02:57.705 ***** 2021-02-20T13:05:00.062 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-02-20T13:05:00.064 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:00.106 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:00 +0000 (0:00:00.050) 0:02:57.755 ***** 2021-02-20T13:05:00.135 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:00.181 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:00 +0000 (0:00:00.074) 0:02:57.830 ***** 2021-02-20T13:05:00.254 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-02-20T13:05:00.256 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T13:05:00.315 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:00 +0000 (0:00:00.133) 0:02:57.964 ***** 2021-02-20T13:05:00.325 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T13:05:00.381 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:00 +0000 (0:00:00.066) 0:02:58.030 ***** 2021-02-20T13:05:00.389 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-02-20T13:05:00.391 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-02-20T13:05:00.433 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:00 +0000 (0:00:00.051) 0:02:58.082 ***** 2021-02-20T13:05:00.440 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-02-20T13:05:00.520 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:00 +0000 (0:00:00.087) 0:02:58.170 ***** 2021-02-20T13:05:01.009 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2021-02-20T13:05:01.010 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"changed": true} 2021-02-20T13:05:01.057 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:01 +0000 (0:00:00.536) 0:02:58.707 ***** 2021-02-20T13:05:01.676 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2021-02-20T13:05:01.678 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-02-20T13:05:01.754 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:01 +0000 (0:00:00.696) 0:02:59.403 ***** 2021-02-20T13:05:01.795 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2021-02-20T13:05:01.797 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:01.844 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:01 +0000 (0:00:00.089) 0:02:59.493 ***** 2021-02-20T13:05:01.877 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2021-02-20T13:05:01.879 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:01.943 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:01 +0000 (0:00:00.099) 0:02:59.592 ***** 2021-02-20T13:05:01.953 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2021-02-20T13:05:01.955 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true} 2021-02-20T13:05:02.060 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:02 +0000 (0:00:00.116) 0:02:59.709 ***** 2021-02-20T13:05:02.076 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-02-20T13:05:02.077 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:02.149 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:02 +0000 (0:00:00.084) 0:02:59.794 ***** 2021-02-20T13:05:02.212 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-02-20T13:05:02.214 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:02.280 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:02 +0000 (0:00:00.134) 0:02:59.929 ***** 2021-02-20T13:05:02.310 INFO:teuthology.task.ansible.out: 2021-02-20T13:05:02.312 INFO:teuthology.task.ansible.out:TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2021-02-20T13:05:02.314 INFO:teuthology.task.ansible.out: changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-02-20T13:05:02.380 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:02 +0000 (0:00:00.099) 0:03:00.029 ***** 2021-02-20T13:05:02.429 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2021-02-20T13:05:02.431 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:02.465 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:02 +0000 (0:00:00.085) 0:03:00.114 ***** 2021-02-20T13:05:02.493 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-02-20T13:05:02.496 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-02-20T13:05:02.538 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:02 +0000 (0:00:00.072) 0:03:00.187 ***** 2021-02-20T13:05:02.545 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2021-02-20T13:05:02.546 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:02.606 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:02 +0000 (0:00:00.067) 0:03:00.255 ***** 2021-02-20T13:05:02.720 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-02-20T13:05:02.722 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:02.768 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:02 +0000 (0:00:00.162) 0:03:00.417 ***** 2021-02-20T13:05:02.778 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-02-20T13:05:02.780 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T13:05:02.842 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:02 +0000 (0:00:00.070) 0:03:00.488 ***** 2021-02-20T13:05:02.848 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-02-20T13:05:02.850 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:02.916 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:02 +0000 (0:00:00.077) 0:03:00.566 ***** 2021-02-20T13:05:03.119 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-02-20T13:05:03.184 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-02-20T13:05:03.186 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:03 +0000 (0:00:00.269) 0:03:00.835 ***** 2021-02-20T13:05:03.426 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-02-20T13:05:03.492 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:03 +0000 (0:00:00.306) 0:03:01.142 ***** 2021-02-20T13:05:03.501 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-02-20T13:05:03.503 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T13:05:03.563 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:03 +0000 (0:00:00.069) 0:03:01.211 ***** 2021-02-20T13:05:04.053 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-02-20T13:05:04.055 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-02-20T13:05:04.056 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-02-20T13:05:04.056 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-02-20T13:05:04.167 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:04 +0000 (0:00:00.605) 0:03:01.817 ***** 2021-02-20T13:05:04.245 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:04 +0000 (0:00:00.077) 0:03:01.894 ***** 2021-02-20T13:05:04.998 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-02-20T13:05:05.053 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:05 +0000 (0:00:00.808) 0:03:02.702 ***** 2021-02-20T13:05:05.598 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-02-20T13:05:05.601 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"cache_update_time": 1613825943, "cache_updated": false, "changed": false} 2021-02-20T13:05:05.672 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:05 +0000 (0:00:00.619) 0:03:03.322 ***** 2021-02-20T13:05:05.915 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-02-20T13:05:05.917 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.046875", "end": "2021-02-20 13:05:05.882856", "rc": 0, "start": "2021-02-20 13:05:05.835981", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2021-02-20T13:05:06.015 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:06 +0000 (0:00:00.342) 0:03:03.664 ***** 2021-02-20T13:05:06.049 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-02-20T13:05:06.051 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"cache_update_time": 1613825899, "cache_updated": false, "changed": false} 2021-02-20T13:05:06.150 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:06 +0000 (0:00:00.126) 0:03:03.791 ***** 2021-02-20T13:05:06.173 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-02-20T13:05:06.175 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:06.256 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:06 +0000 (0:00:00.114) 0:03:03.905 ***** 2021-02-20T13:05:06.305 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:06.375 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:06 +0000 (0:00:00.118) 0:03:04.024 ***** 2021-02-20T13:05:06.397 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-02-20T13:05:06.400 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.039198", "end": "2021-02-20 13:05:06.327692", "rc": 0, "start": "2021-02-20 13:05:06.288494", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2021-02-20T13:05:06.496 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:06 +0000 (0:00:00.121) 0:03:04.146 ***** 2021-02-20T13:05:06.571 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-02-20T13:05:06.573 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:06.660 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:06 +0000 (0:00:00.163) 0:03:04.309 ***** 2021-02-20T13:05:06.721 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:06.793 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:06 +0000 (0:00:00.132) 0:03:04.442 ***** 2021-02-20T13:05:07.347 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-02-20T13:05:07.349 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"cache_update_time": 1613825943, "cache_updated": false, "changed": false} 2021-02-20T13:05:07.400 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:07 +0000 (0:00:00.607) 0:03:05.050 ***** 2021-02-20T13:05:07.474 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2021-02-20T13:05:07.476 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:07.538 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:07 +0000 (0:00:00.137) 0:03:05.187 ***** 2021-02-20T13:05:07.795 INFO:teuthology.task.ansible.out: 2021-02-20T13:05:07.797 INFO:teuthology.task.ansible.out:TASK [testnode : Install ntp package on apt based systems.] ******************** ok: [smithi170.front.sepia.ceph.com] => {"cache_update_time": 1613825899, "cache_updated": false, "changed": false} 2021-02-20T13:05:07.863 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:07 +0000 (0:00:00.323) 0:03:05.511 ***** 2021-02-20T13:05:07.889 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2021-02-20T13:05:07.891 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:07.941 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:07 +0000 (0:00:00.079) 0:03:05.590 ***** 2021-02-20T13:05:08.022 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-02-20T13:05:08.024 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-02-20T13:05:08.074 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:08 +0000 (0:00:00.132) 0:03:05.723 ***** 2021-02-20T13:05:08.124 INFO:teuthology.task.ansible.out: 2021-02-20T13:05:08.126 INFO:teuthology.task.ansible.out:TASK [testnode : Create the chrony.conf file] ********************************** skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:08.163 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:08 +0000 (0:00:00.088) 0:03:05.812 ***** 2021-02-20T13:05:08.383 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-02-20T13:05:08.385 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-02-20T13:05:08.429 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:08 +0000 (0:00:00.266) 0:03:06.079 ***** 2021-02-20T13:05:08.469 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-02-20T13:05:08.470 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-02-20 13:01:48 UTC", "ActiveEnterTimestampMonotonic": "28425425", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "-.mount system.slice sysinit.target network.target systemd-journald.socket basic.target systemd-tmpfiles-setup.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-02-20 13:01:41 UTC", "AssertTimestampMonotonic": "21972065", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-02-20 13:01:41 UTC", "ConditionTimestampMonotonic": "21972064", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2021-02-20T13:05:08.471 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1771", "ExecMainStartTimestamp": "Sat 2021-02-20 13:01:48 UTC", "ExecMainStartTimestampMonotonic": "28425359", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-02-20 13:01:41 UTC", "InactiveExitTimestampMonotonic": "21972880", "InvocationID": "ff45bc95232e4ae28b9cfc2bb408f936", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2021-02-20T13:05:08.473 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1771", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount system.slice sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 13:01:48 UTC", "S 2021-02-20T13:05:08.475 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "28425425", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-02-20 13:01:48 UTC", "WatchdogTimestampMonotonic": "28425422", "WatchdogUSec": "0"}} 2021-02-20T13:05:08.531 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:08 +0000 (0:00:00.102) 0:03:06.181 ***** 2021-02-20T13:05:08.540 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-02-20T13:05:08.542 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:08.589 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:08 +0000 (0:00:00.057) 0:03:06.239 ***** 2021-02-20T13:05:08.632 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T13:05:08.635 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-02-20T13:05:08.684 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:08 +0000 (0:00:00.094) 0:03:06.333 ***** 2021-02-20T13:05:08.765 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T13:05:08.766 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:08.814 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:08 +0000 (0:00:00.129) 0:03:06.463 ***** 2021-02-20T13:05:08.816 INFO:teuthology.task.ansible.out: 2021-02-20T13:05:08.881 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-02-20T13:05:08.883 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:08.929 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:08 +0000 (0:00:00.115) 0:03:06.579 ***** 2021-02-20T13:05:08.940 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-02-20T13:05:08.943 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-02-20 13:01:37 UTC", "ActiveEnterTimestampMonotonic": "24932302", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target system.slice -.mount network.target sysinit.target systemd-tmpfiles-setup.service systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-02-20 13:01:31 UTC", "AssertTimestampMonotonic": "18552070", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-02-20 13:01:31 UTC", "ConditionTimestampMonotonic": "18552070", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2021-02-20T13:05:08.944 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1806", "ExecMainStartTimestamp": "Sat 2021-02-20 13:01:37 UTC", "ExecMainStartTimestampMonotonic": "24932279", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-02-20 13:01:31 UTC", "InactiveExitTimestampMonotonic": "18552771", "InvocationID": "4566cb0b721e432db5acdc92bf4e45ec", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2021-02-20T13:05:08.946 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1806", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount system.slice sysinit.target", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 13:01:37 UTC", "S 2021-02-20T13:05:08.947 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "24932302", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-02-20 13:01:37 UTC", "WatchdogTimestampMonotonic": "24932300", "WatchdogUSec": "0"}} 2021-02-20T13:05:09.231 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:09 +0000 (0:00:00.301) 0:03:06.880 ***** 2021-02-20T13:05:09.298 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T13:05:09.300 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-02-20T13:05:09.392 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:09 +0000 (0:00:00.161) 0:03:07.042 ***** 2021-02-20T13:05:09.485 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T13:05:09.487 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:09.559 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:09 +0000 (0:00:00.166) 0:03:07.208 ***** 2021-02-20T13:05:09.619 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-02-20T13:05:09.621 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:09.693 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:09 +0000 (0:00:00.133) 0:03:07.342 ***** 2021-02-20T13:05:09.857 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-02-20T13:05:09.859 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"cache_update_time": 1613825943, "cache_updated": false, "changed": false} 2021-02-20T13:05:09.924 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:09 +0000 (0:00:00.222) 0:03:07.565 ***** 2021-02-20T13:05:09.978 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:10.039 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:10 +0000 (0:00:00.123) 0:03:07.689 ***** 2021-02-20T13:05:10.311 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-02-20T13:05:10.312 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-02-20T13:05:10.362 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:10 +0000 (0:00:00.323) 0:03:08.012 ***** 2021-02-20T13:05:10.649 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-02-20T13:05:10.651 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"cache_update_time": 1613825899, "cache_updated": false, "changed": false} 2021-02-20T13:05:10.735 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:10 +0000 (0:00:00.372) 0:03:08.385 ***** 2021-02-20T13:05:10.820 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-02-20T13:05:10.822 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:10.882 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:10 +0000 (0:00:00.147) 0:03:08.532 ***** 2021-02-20T13:05:10.945 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-02-20T13:05:10.947 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-02-20T13:05:11.007 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:11 +0000 (0:00:00.124) 0:03:08.657 ***** 2021-02-20T13:05:11.073 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-02-20T13:05:11.075 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-02-20T13:05:11.114 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:11 +0000 (0:00:00.106) 0:03:08.763 ***** 2021-02-20T13:05:11.565 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-02-20T13:05:11.570 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-02-20T13:05:11.638 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:11 +0000 (0:00:00.524) 0:03:09.287 ***** 2021-02-20T13:05:12.063 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-02-20T13:05:12.102 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:12 +0000 (0:00:00.463) 0:03:09.752 ***** 2021-02-20T13:05:12.476 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-02-20T13:05:12.516 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:12 +0000 (0:00:00.413) 0:03:10.165 ***** 2021-02-20T13:05:12.661 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-02-20T13:05:12.663 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false} 2021-02-20T13:05:12.706 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:12 +0000 (0:00:00.189) 0:03:10.355 ***** 2021-02-20T13:05:12.758 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ ok: [smithi183.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2021-02-20T13:05:12.794 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:12 +0000 (0:00:00.087) 0:03:10.443 ***** 2021-02-20T13:05:12.967 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-02-20T13:05:12.968 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1613825877.750703, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1613825877.7467031, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 746, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1613825877.7467031, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "3375002756", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-02-20T13:05:13.004 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:13 +0000 (0:00:00.210) 0:03:10.654 ***** 2021-02-20T13:05:13.077 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-02-20T13:05:13.079 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false} 2021-02-20T13:05:13.152 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:13 +0000 (0:00:00.148) 0:03:10.802 ***** 2021-02-20T13:05:13.209 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2021-02-20T13:05:13.274 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:13 +0000 (0:00:00.121) 0:03:10.923 ***** 2021-02-20T13:05:13.402 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-02-20T13:05:13.404 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-02-20T13:05:13.446 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:13 +0000 (0:00:00.172) 0:03:11.096 ***** 2021-02-20T13:05:13.473 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-02-20T13:05:13.475 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1613825879.2505562, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1613825879.246556, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 746, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1613825879.246556, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "3375002756", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-02-20T13:05:13.518 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:13 +0000 (0:00:00.071) 0:03:11.167 ***** 2021-02-20T13:05:13.545 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-02-20T13:05:13.547 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-02-20T13:05:13.591 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:13 +0000 (0:00:00.073) 0:03:11.241 ***** 2021-02-20T13:05:13.641 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-02-20T13:05:13.642 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:13.684 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:13 +0000 (0:00:00.093) 0:03:11.334 ***** 2021-02-20T13:05:13.695 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-02-20T13:05:13.697 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-02-20T13:05:13.765 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:13 +0000 (0:00:00.081) 0:03:11.415 ***** 2021-02-20T13:05:13.803 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-02-20T13:05:13.805 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-02-20T13:05:13.867 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:13 +0000 (0:00:00.102) 0:03:11.517 ***** 2021-02-20T13:05:13.893 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2021-02-20T13:05:13.895 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-02-20T13:05:13.949 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:13 +0000 (0:00:00.081) 0:03:11.598 ***** 2021-02-20T13:05:13.956 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-02-20T13:05:13.958 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:14.004 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:14 +0000 (0:00:00.055) 0:03:11.653 ***** 2021-02-20T13:05:14.121 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-02-20T13:05:14.123 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-02-20T13:05:14.192 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:14 +0000 (0:00:00.188) 0:03:11.842 ***** 2021-02-20T13:05:14.228 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2021-02-20T13:05:14.230 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-02-20T13:05:14.305 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:14 +0000 (0:00:00.113) 0:03:11.955 ***** 2021-02-20T13:05:14.318 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-02-20T13:05:14.320 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:14.372 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:14 +0000 (0:00:00.066) 0:03:12.022 ***** 2021-02-20T13:05:14.474 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-02-20T13:05:14.476 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-02-20T13:05:14.486 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2021-02-20T13:05:14.488 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-02-20T13:05:14.498 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2021-02-20T13:05:14.553 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:14 +0000 (0:00:00.180) 0:03:12.203 ***** 2021-02-20T13:05:14.578 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-02-20T13:05:14.580 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-02-20T13:05:14.627 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:14 +0000 (0:00:00.073) 0:03:12.276 ***** 2021-02-20T13:05:14.658 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** skipping: [smithi183.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi183.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:14.679 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:14.681 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** skipping: [smithi183.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:14.732 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:14 +0000 (0:00:00.105) 0:03:12.382 ***** 2021-02-20T13:05:14.743 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:14.745 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:14.801 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:14 +0000 (0:00:00.069) 0:03:12.451 ***** 2021-02-20T13:05:14.823 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-02-20T13:05:14.825 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2021-02-20T13:05:14.834 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-02-20T13:05:14.842 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [smithi170.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-02-20T13:05:14.913 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:14 +0000 (0:00:00.111) 0:03:12.562 ***** 2021-02-20T13:05:14.926 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2021-02-20T13:05:14.988 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:14 +0000 (0:00:00.075) 0:03:12.638 ***** 2021-02-20T13:05:15.007 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-02-20T13:05:15.009 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:15.076 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:15 +0000 (0:00:00.087) 0:03:12.725 ***** 2021-02-20T13:05:15.090 INFO:teuthology.task.ansible.out: 2021-02-20T13:05:15.096 INFO:teuthology.task.ansible.out:TASK [testnode : Define net_to_configure var] ********************************** 2021-02-20T13:05:15.097 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:15.097 INFO:teuthology.task.ansible.out: 2021-02-20T13:05:15.110 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:15.113 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:15.179 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:15 +0000 (0:00:00.103) 0:03:12.829 ***** 2021-02-20T13:05:15.203 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-02-20T13:05:15.205 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:15.285 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:15 +0000 (0:00:00.105) 0:03:12.934 ***** 2021-02-20T13:05:15.293 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T13:05:15.298 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:15.349 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:15 +0000 (0:00:00.063) 0:03:12.998 ***** 2021-02-20T13:05:15.371 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-02-20T13:05:15.373 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:15.409 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:15 +0000 (0:00:00.059) 0:03:13.058 ***** 2021-02-20T13:05:15.419 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-02-20T13:05:15.421 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:15.473 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:15 +0000 (0:00:00.063) 0:03:13.122 ***** 2021-02-20T13:05:15.487 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-02-20T13:05:15.489 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:15.531 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:15 +0000 (0:00:00.057) 0:03:13.180 ***** 2021-02-20T13:05:15.542 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-02-20T13:05:15.543 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:15.585 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:15 +0000 (0:00:00.054) 0:03:13.235 ***** 2021-02-20T13:05:15.594 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-02-20T13:05:15.596 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:15.677 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:15 +0000 (0:00:00.091) 0:03:13.327 ***** 2021-02-20T13:05:15.684 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-02-20T13:05:15.686 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:15.738 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:15 +0000 (0:00:00.060) 0:03:13.387 ***** 2021-02-20T13:05:15.769 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-02-20T13:05:15.770 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:15.835 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:15 +0000 (0:00:00.097) 0:03:13.484 ***** 2021-02-20T13:05:15.845 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-02-20T13:05:15.847 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:15.930 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:15 +0000 (0:00:00.094) 0:03:13.579 ***** 2021-02-20T13:05:15.940 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:16.000 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:15 +0000 (0:00:00.070) 0:03:13.650 ***** 2021-02-20T13:05:16.008 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-02-20T13:05:16.010 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:16.080 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:16 +0000 (0:00:00.079) 0:03:13.729 ***** 2021-02-20T13:05:16.146 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-02-20T13:05:16.148 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:16.244 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:16 +0000 (0:00:00.164) 0:03:13.893 ***** 2021-02-20T13:05:16.252 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-02-20T13:05:16.254 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.011042", "end": "2021-02-20 13:05:16.144986", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-02-20 13:05:16.133944", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:05:16.289 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:16.334 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:16 +0000 (0:00:00.090) 0:03:13.984 ***** 2021-02-20T13:05:16.344 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-02-20T13:05:16.346 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006139", "end": "2021-02-20 13:05:16.291417", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-02-20 13:05:16.285278", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:05:16.458 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005339", "end": "2021-02-20 13:05:16.419275", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-02-20 13:05:16.413936", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-02-20T13:05:16.507 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.007132", "end": "2021-02-20 13:05:16.464310", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-02-20 13:05:16.457178", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:05:16.615 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.027674", "end": "2021-02-20 13:05:16.585447", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-02-20 13:05:16.557773", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:05:16.666 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:16 +0000 (0:00:00.328) 0:03:14.312 ***** 2021-02-20T13:05:16.672 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006082", "end": "2021-02-20 13:05:16.600421", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-02-20 13:05:16.594339", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:05:16.714 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-02-20T13:05:16.750 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:16 +0000 (0:00:00.087) 0:03:14.399 ***** 2021-02-20T13:05:16.759 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-02-20T13:05:16.761 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005579", "end": "2021-02-20 13:05:16.725663", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-02-20 13:05:16.720084", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-02-20T13:05:16.882 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.019201", "end": "2021-02-20 13:05:16.854405", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-02-20 13:05:16.835204", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:05:16.934 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:16 +0000 (0:00:00.183) 0:03:14.583 ***** 2021-02-20T13:05:16.944 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** changed: [smithi183.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-02-20T13:05:16.983 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-02-20T13:05:17.026 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:17 +0000 (0:00:00.091) 0:03:14.675 ***** 2021-02-20T13:05:17.188 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-02-20T13:05:17.190 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-02-20T13:05:17.308 INFO:teuthology.task.ansible.out: PLAY [testnodes,senta,vossi,jenkins_slaves] ************************************ 2021-02-20T13:05:17.380 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:17 +0000 (0:00:00.354) 0:03:15.029 ***** 2021-02-20T13:05:17.433 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:17 +0000 (0:00:00.052) 0:03:15.082 ***** 2021-02-20T13:05:18.156 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi170.front.sepia.ceph.com] 2021-02-20T13:05:18.255 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:18 +0000 (0:00:00.822) 0:03:15.905 ***** 2021-02-20T13:05:18.270 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] 2021-02-20T13:05:18.324 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-20T13:05:18.325 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-02-20T13:05:18.363 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:18 +0000 (0:00:00.106) 0:03:16.011 ***** 2021-02-20T13:05:18.413 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:18 +0000 (0:00:00.049) 0:03:16.061 ***** 2021-02-20T13:05:18.425 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-20T13:05:18.427 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:18.474 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:18 +0000 (0:00:00.062) 0:03:16.123 ***** 2021-02-20T13:05:18.489 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** ok: [smithi183.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-02-20T13:05:18.537 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:18 +0000 (0:00:00.063) 0:03:16.186 ***** 2021-02-20T13:05:18.545 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-02-20T13:05:18.547 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-02-20T13:05:18.590 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:18 +0000 (0:00:00.052) 0:03:16.239 ***** 2021-02-20T13:05:18.601 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-20T13:05:18.602 INFO:teuthology.task.ansible.out:skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:18.655 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:18 +0000 (0:00:00.065) 0:03:16.304 ***** 2021-02-20T13:05:18.706 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-02-20T13:05:18.707 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-02-20T13:05:18.749 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:18 +0000 (0:00:00.093) 0:03:16.398 ***** 2021-02-20T13:05:19.519 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** ok: [smithi170.front.sepia.ceph.com] => {"cache_update_time": 1613825899, "cache_updated": false, "changed": false} 2021-02-20T13:05:19.560 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:19 +0000 (0:00:00.810) 0:03:17.209 ***** 2021-02-20T13:05:19.599 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-20T13:05:19.601 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:19.641 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:19 +0000 (0:00:00.081) 0:03:17.290 ***** 2021-02-20T13:05:19.699 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-20T13:05:19.701 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-02-20T13:05:19.746 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:19 +0000 (0:00:00.105) 0:03:17.396 ***** 2021-02-20T13:05:19.754 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** ok: [smithi183.front.sepia.ceph.com] => {"cache_update_time": 1613825943, "cache_updated": false, "changed": false} 2021-02-20T13:05:19.819 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:19 +0000 (0:00:00.066) 0:03:17.463 ***** 2021-02-20T13:05:19.872 INFO:teuthology.task.ansible.out: 2021-02-20T13:05:19.874 INFO:teuthology.task.ansible.out:TASK [container-host : set_fact] *********************************************** skipping: [smithi183.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T13:05:19.918 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:19 +0000 (0:00:00.104) 0:03:17.567 ***** 2021-02-20T13:05:19.945 INFO:teuthology.task.ansible.out: 2021-02-20T13:05:19.947 INFO:teuthology.task.ansible.out:TASK [container-host : Create container_mirror_cert_paths] ********************* changed: [smithi170.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T13:05:19.957 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-20T13:05:19.959 INFO:teuthology.task.ansible.out:ok: [smithi183.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-02-20T13:05:20.001 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:19 +0000 (0:00:00.083) 0:03:17.650 ***** 2021-02-20T13:05:20.074 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-02-20T13:05:20.076 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T13:05:20.131 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:20 +0000 (0:00:00.129) 0:03:17.780 ***** 2021-02-20T13:05:20.213 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T13:05:20.341 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T13:05:20.388 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:20 +0000 (0:00:00.257) 0:03:18.037 ***** 2021-02-20T13:05:20.694 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2021-02-20T13:05:20.722 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613826320.2046041-90771464217714/source", "state": "file", "uid": 0} 2021-02-20T13:05:21.130 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613826320.4358845-244716400491498/source", "state": "file", "uid": 0} 2021-02-20T13:05:21.247 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613826320.6680846-246110509498386/source", "state": "file", "uid": 0} 2021-02-20T13:05:21.296 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:21 +0000 (0:00:00.907) 0:03:18.945 ***** 2021-02-20T13:05:21.613 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613826321.1179411-180104308346842/source", "state": "file", "uid": 0} 2021-02-20T13:05:21.699 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:21 +0000 (0:00:00.403) 0:03:19.348 ***** 2021-02-20T13:05:24.850 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-02-20T13:05:24.852 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-0Glnf8-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-0Glnf8-build", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packag 2021-02-20T13:05:24.853 INFO:teuthology.task.ansible.out:es/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} 2021-02-20T13:05:24.925 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:24 +0000 (0:00:03.225) 0:03:22.574 ***** 2021-02-20T13:05:25.176 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-02-20T13:05:25.179 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-02-20T13:05:25.249 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:25 +0000 (0:00:00.324) 0:03:22.899 ***** 2021-02-20T13:05:25.340 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-02-20T13:05:25.342 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-1wBPjK-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-1wBPjK-build", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packag 2021-02-20T13:05:25.344 INFO:teuthology.task.ansible.out:es/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} 2021-02-20T13:05:25.399 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:25 +0000 (0:00:00.149) 0:03:23.048 ***** 2021-02-20T13:05:25.655 INFO:teuthology.task.ansible.out: 2021-02-20T13:05:25.657 INFO:teuthology.task.ansible.out:TASK [container-host : Check for docker's daemon.json] ************************* ok: [smithi183.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-02-20T13:05:25.718 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:25 +0000 (0:00:00.318) 0:03:23.367 ***** 2021-02-20T13:05:25.897 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ 2021-02-20T13:05:25.899 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613826325.3422828-180868603163452/source", "state": "file", "uid": 0} 2021-02-20T13:05:25.984 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:25 +0000 (0:00:00.266) 0:03:23.634 ***** 2021-02-20T13:05:26.347 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2021-02-20T13:05:26.349 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.119888", "end": "2021-02-20 13:05:26.305511", "rc": 0, "start": "2021-02-20 13:05:26.185623", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:05:26.411 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:26 +0000 (0:00:00.426) 0:03:24.060 ***** 2021-02-20T13:05:26.433 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ 2021-02-20T13:05:26.435 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613826325.7856157-231835128531146/source", "state": "file", "uid": 0} 2021-02-20T13:05:26.499 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:26 +0000 (0:00:00.088) 0:03:24.149 ***** 2021-02-20T13:05:26.811 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2021-02-20T13:05:26.812 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.121142", "end": "2021-02-20 13:05:26.790759", "rc": 0, "start": "2021-02-20 13:05:26.669617", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T13:05:26.865 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 13:05:26 +0000 (0:00:00.365) 0:03:24.514 ***** 2021-02-20T13:05:29.767 INFO:teuthology.task.ansible.out: TASK [container-host : Restart docker service] ********************************* 2021-02-20T13:05:29.771 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "sysinit.target network-online.target docker.socket containerd.service firewalld.service system.slice systemd-journald.socket basic.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "yes", "DelegateControllers": "cpu cpuacct io blkio memory devices pids", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMa 2021-02-20T13:05:29.773 INFO:teuthology.task.ansible.out:inStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "12802 2021-02-20T13:05:29.774 INFO:teuthology.task.ansible.out:9", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "docker.socket system.slice sysinit.target", "Restart": "always", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 13:04:12 UTC", "StateChangeTimestampMonotonic": "179707400", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "S 2021-02-20T13:05:29.776 INFO:teuthology.task.ansible.out:uccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-online.target containerd.service", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-02-20T13:05:30.128 INFO:teuthology.task.ansible.out:changed: [smithi183.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "systemd-journald.socket firewalld.service system.slice network-online.target docker.socket sysinit.target basic.target containerd.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "yes", "DelegateControllers": "cpu cpuacct io blkio memory devices pids", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMa 2021-02-20T13:05:30.130 INFO:teuthology.task.ansible.out:inStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "12802 2021-02-20T13:05:30.131 INFO:teuthology.task.ansible.out:9", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target docker.socket system.slice", "Restart": "always", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 13:04:13 UTC", "StateChangeTimestampMonotonic": "173409753", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "S 2021-02-20T13:05:30.133 INFO:teuthology.task.ansible.out:uccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "containerd.service network-online.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-02-20T13:05:30.241 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2021-02-20T13:05:30.246 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** 2021-02-20T13:05:30.248 INFO:teuthology.task.ansible.out:skipping: no hosts matched 2021-02-20T13:05:30.250 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2021-02-20T13:05:30.252 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* smithi170.front.sepia.ceph.com : ok=126 changed=38 unreachable=0 failed=0 skipped=184 rescued=0 ignored=0 smithi183.front.sepia.ceph.com : ok=126 changed=38 unreachable=0 failed=0 skipped=184 rescued=0 ignored=0 Saturday 20 February 2021 13:05:30 +0000 (0:00:03.386) 0:03:27.900 ***** =============================================================================== users : Create all admin users with sudo access. ----------------------- 26.59s users : Update authorized_keys using the keys repo --------------------- 26.05s testnode : Ensure packages are not present. ----------------------------- 8.79s users : Remove revoked users -------------------------------------------- 8.71s testnode : Install packages --------------------------------------------- 8.68s testnode : Zap all non-root disks --------------------------------------- 8.67s common : Update apt cache ----------------------------------------------- 3.71s container-host : Restart docker service --------------------------------- 3.39s container-host : Install registries-conf-ctl ---------------------------- 3.23s testnode : Update apt cache. -------------------------------------------- 2.87s users : Update authorized_keys for each user with literal keys ---------- 2.68s Gathering Facts --------------------------------------------------------- 2.59s testnode : Install apt keys --------------------------------------------- 2.25s testnode : Blow away lingering OSD data and FSIDs ----------------------- 1.98s users : Clone the keys repo --------------------------------------------- 1.63s testnode : Create logical volume(s) ------------------------------------- 1.48s common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/. --- 1.48s common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/. --- 1.45s common : Make s 2021-02-20T13:05:30.253 INFO:teuthology.task.ansible.out:ure smartmontools is installed --------------------------- 1.22s ansible-managed : Add authorized keys for the ansible user. ------------- 1.13s 2021-02-20T13:05:30.648 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi170.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com')] 2021-02-20T13:05:30.649 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-20T13:05:30.651 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T13:05:30.725 DEBUG:teuthology.orchestra.run.smithi170:> true 2021-02-20T13:05:30.928 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-20T13:05:30.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T13:05:31.050 DEBUG:teuthology.orchestra.run.smithi183:> true 2021-02-20T13:05:31.255 INFO:teuthology.run_tasks:Running task clock... 2021-02-20T13:05:31.269 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2021-02-20T13:05:31.270 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-02-20T13:05:31.271 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-02-20T13:05:31.325 INFO:teuthology.orchestra.run.smithi170.stdout:20 Feb 13:05:31 ntpd[8366]: ntpd 4.2.8p10@1.3728-o (1): Starting 2021-02-20T13:05:31.326 INFO:teuthology.orchestra.run.smithi170.stdout:20 Feb 13:05:31 ntpd[8366]: Command line: ntpd -gq 2021-02-20T13:05:31.328 INFO:teuthology.orchestra.run.smithi170.stdout:20 Feb 13:05:31 ntpd[8366]: proto: precision = 0.133 usec (-23) 2021-02-20T13:05:31.328 INFO:teuthology.orchestra.run.smithi170.stdout:20 Feb 13:05:31 ntpd[8366]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-02-20T13:05:31.328 INFO:teuthology.orchestra.run.smithi170.stdout:20 Feb 13:05:31 ntpd[8366]: restrict ::: KOD does nothing without LIMITED. 2021-02-20T13:05:31.328 INFO:teuthology.orchestra.run.smithi170.stdout:20 Feb 13:05:31 ntpd[8366]: Listen and drop on 0 v6wildcard [::]:123 2021-02-20T13:05:31.329 INFO:teuthology.orchestra.run.smithi170.stdout:20 Feb 13:05:31 ntpd[8366]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-02-20T13:05:31.329 INFO:teuthology.orchestra.run.smithi170.stdout:20 Feb 13:05:31 ntpd[8366]: Listen normally on 2 lo 127.0.0.1:123 2021-02-20T13:05:31.329 INFO:teuthology.orchestra.run.smithi170.stdout:20 Feb 13:05:31 ntpd[8366]: Listen normally on 3 enp3s0f1 172.21.15.170:123 2021-02-20T13:05:31.329 INFO:teuthology.orchestra.run.smithi170.stdout:20 Feb 13:05:31 ntpd[8366]: Listen normally on 4 lo [::1]:123 2021-02-20T13:05:31.329 INFO:teuthology.orchestra.run.smithi170.stdout:20 Feb 13:05:31 ntpd[8366]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7255%5]:123 2021-02-20T13:05:31.330 INFO:teuthology.orchestra.run.smithi170.stdout:20 Feb 13:05:31 ntpd[8366]: Listening on routing socket on fd #22 for interface updates 2021-02-20T13:05:31.330 INFO:teuthology.orchestra.run.smithi170.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-02-20T13:05:31.331 INFO:teuthology.orchestra.run.smithi170.stderr:restrict ::: KOD does nothing without LIMITED. 2021-02-20T13:05:33.326 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:33 ntpd[8366]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T13:05:33.373 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:33 ntpd[8366]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T13:05:33.373 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:33 ntpd[8366]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T13:05:33.374 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:33 ntpd[8366]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T13:05:34.326 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:34 ntpd[8366]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T13:05:34.326 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:34 ntpd[8366]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T13:05:34.327 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:34 ntpd[8366]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T13:05:34.327 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:34 ntpd[8366]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T13:05:35.327 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:35 ntpd[8366]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T13:05:35.328 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:35 ntpd[8366]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T13:05:35.328 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:35 ntpd[8366]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T13:05:35.329 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:35 ntpd[8366]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T13:05:35.329 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:35 ntpd[8366]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T13:05:35.330 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:35 ntpd[8366]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T13:05:35.330 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:35 ntpd[8366]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T13:05:35.331 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:35 ntpd[8366]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T13:05:36.329 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:36 ntpd[8366]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T13:05:36.329 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:36 ntpd[8366]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T13:05:36.329 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:36 ntpd[8366]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T13:05:36.330 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:36 ntpd[8366]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T13:05:36.330 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:36 ntpd[8366]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T13:05:36.330 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:36 ntpd[8366]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T13:05:36.330 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:36 ntpd[8366]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T13:05:36.331 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:36 ntpd[8366]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T13:05:37.328 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:37 ntpd[8366]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T13:05:37.329 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:37 ntpd[8366]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T13:05:37.329 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:37 ntpd[8366]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T13:05:37.329 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:37 ntpd[8366]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T13:05:37.329 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:37 ntpd[8366]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T13:05:37.330 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:37 ntpd[8366]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T13:05:37.330 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:37 ntpd[8366]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T13:05:37.330 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:37 ntpd[8366]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T13:05:38.326 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:38 ntpd[8366]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T13:05:38.327 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:38 ntpd[8366]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T13:05:38.327 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:38 ntpd[8366]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T13:05:38.327 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:38 ntpd[8366]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T13:05:38.327 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:38 ntpd[8366]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T13:05:38.328 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:38 ntpd[8366]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T13:05:38.328 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:38 ntpd[8366]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T13:05:38.328 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:38 ntpd[8366]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T13:05:39.326 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:39 ntpd[8366]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T13:05:39.327 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:39 ntpd[8366]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T13:05:39.327 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:39 ntpd[8366]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T13:05:39.327 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:39 ntpd[8366]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T13:05:39.328 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:39 ntpd[8366]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T13:05:39.328 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:39 ntpd[8366]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T13:05:39.328 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:39 ntpd[8366]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T13:05:39.328 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:39 ntpd[8366]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T13:05:39.328 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:39 ntpd[8366]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-02-20T13:05:39.329 INFO:teuthology.orchestra.run.smithi170.stderr:20 Feb 13:05:39 ntpd[8366]: can't open /var/log/ntpstats/loopstats.20210220: Permission denied 2021-02-20T13:05:39.329 INFO:teuthology.orchestra.run.smithi170.stdout:20 Feb 13:05:39 ntpd[8366]: ntpd: time slew -0.000968 s 2021-02-20T13:05:39.329 INFO:teuthology.orchestra.run.smithi170.stdout:ntpd: time slew -0.000968s 2021-02-20T13:05:39.370 INFO:teuthology.orchestra.run.smithi170.stdout: remote refid st t when poll reach delay offset jitter 2021-02-20T13:05:39.485 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================== 2021-02-20T13:05:39.486 INFO:teuthology.orchestra.run.smithi170.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T13:05:39.486 INFO:teuthology.orchestra.run.smithi170.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T13:05:39.487 INFO:teuthology.orchestra.run.smithi170.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T13:05:39.487 INFO:teuthology.orchestra.run.smithi170.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T13:05:39.488 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-02-20T13:05:39.489 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-02-20T13:05:39.559 INFO:teuthology.orchestra.run.smithi183.stdout:20 Feb 13:05:39 ntpd[8341]: ntpd 4.2.8p10@1.3728-o (1): Starting 2021-02-20T13:05:39.560 INFO:teuthology.orchestra.run.smithi183.stdout:20 Feb 13:05:39 ntpd[8341]: Command line: ntpd -gq 2021-02-20T13:05:39.561 INFO:teuthology.orchestra.run.smithi183.stdout:20 Feb 13:05:39 ntpd[8341]: proto: precision = 0.123 usec (-23) 2021-02-20T13:05:39.561 INFO:teuthology.orchestra.run.smithi183.stdout:20 Feb 13:05:39 ntpd[8341]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-02-20T13:05:39.561 INFO:teuthology.orchestra.run.smithi183.stdout:20 Feb 13:05:39 ntpd[8341]: restrict ::: KOD does nothing without LIMITED. 2021-02-20T13:05:39.562 INFO:teuthology.orchestra.run.smithi183.stdout:20 Feb 13:05:39 ntpd[8341]: Listen and drop on 0 v6wildcard [::]:123 2021-02-20T13:05:39.562 INFO:teuthology.orchestra.run.smithi183.stdout:20 Feb 13:05:39 ntpd[8341]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-02-20T13:05:39.562 INFO:teuthology.orchestra.run.smithi183.stdout:20 Feb 13:05:39 ntpd[8341]: Listen normally on 2 lo 127.0.0.1:123 2021-02-20T13:05:39.562 INFO:teuthology.orchestra.run.smithi183.stdout:20 Feb 13:05:39 ntpd[8341]: Listen normally on 3 enp3s0f1 172.21.15.183:123 2021-02-20T13:05:39.563 INFO:teuthology.orchestra.run.smithi183.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-02-20T13:05:39.563 INFO:teuthology.orchestra.run.smithi183.stderr:restrict ::: KOD does nothing without LIMITED. 2021-02-20T13:05:39.564 INFO:teuthology.orchestra.run.smithi183.stdout:20 Feb 13:05:39 ntpd[8341]: Listen normally on 4 lo [::1]:123 2021-02-20T13:05:39.564 INFO:teuthology.orchestra.run.smithi183.stdout:20 Feb 13:05:39 ntpd[8341]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8085%5]:123 2021-02-20T13:05:39.564 INFO:teuthology.orchestra.run.smithi183.stdout:20 Feb 13:05:39 ntpd[8341]: Listening on routing socket on fd #22 for interface updates 2021-02-20T13:05:40.560 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:40 ntpd[8341]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T13:05:40.561 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:40 ntpd[8341]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T13:05:40.561 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:40 ntpd[8341]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T13:05:40.561 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:40 ntpd[8341]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T13:05:40.562 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:40 ntpd[8341]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T13:05:40.562 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:40 ntpd[8341]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T13:05:40.562 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:40 ntpd[8341]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T13:05:40.562 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:40 ntpd[8341]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T13:05:42.560 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:42 ntpd[8341]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T13:05:42.561 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:42 ntpd[8341]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T13:05:42.561 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:42 ntpd[8341]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T13:05:42.562 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:42 ntpd[8341]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T13:05:42.562 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:42 ntpd[8341]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T13:05:42.562 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:42 ntpd[8341]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T13:05:42.562 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:42 ntpd[8341]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T13:05:42.563 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:42 ntpd[8341]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T13:05:44.561 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:44 ntpd[8341]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T13:05:44.561 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:44 ntpd[8341]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T13:05:44.562 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:44 ntpd[8341]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T13:05:44.562 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:44 ntpd[8341]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T13:05:44.562 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:44 ntpd[8341]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T13:05:44.562 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:44 ntpd[8341]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T13:05:44.562 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:44 ntpd[8341]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T13:05:44.563 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:44 ntpd[8341]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T13:05:44.563 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:44 ntpd[8341]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T13:05:44.563 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:44 ntpd[8341]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T13:05:44.563 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:44 ntpd[8341]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T13:05:44.564 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:44 ntpd[8341]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T13:05:44.564 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:44 ntpd[8341]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T13:05:44.564 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:44 ntpd[8341]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T13:05:44.564 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:44 ntpd[8341]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T13:05:44.564 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:44 ntpd[8341]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T13:05:46.561 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:46 ntpd[8341]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T13:05:46.561 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:46 ntpd[8341]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T13:05:46.561 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:46 ntpd[8341]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T13:05:46.562 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:46 ntpd[8341]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T13:05:46.562 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:46 ntpd[8341]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-02-20T13:05:46.562 INFO:teuthology.orchestra.run.smithi183.stderr:20 Feb 13:05:46 ntpd[8341]: can't open /var/log/ntpstats/loopstats.20210220: Permission denied 2021-02-20T13:05:46.562 INFO:teuthology.orchestra.run.smithi183.stdout:20 Feb 13:05:46 ntpd[8341]: ntpd: time slew -0.002118 s 2021-02-20T13:05:46.563 INFO:teuthology.orchestra.run.smithi183.stdout:ntpd: time slew -0.002118s 2021-02-20T13:05:46.619 INFO:teuthology.orchestra.run.smithi183.stdout: remote refid st t when poll reach delay offset jitter 2021-02-20T13:05:46.619 INFO:teuthology.orchestra.run.smithi183.stdout:============================================================================== 2021-02-20T13:05:46.620 INFO:teuthology.orchestra.run.smithi183.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T13:05:46.620 INFO:teuthology.orchestra.run.smithi183.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T13:05:46.620 INFO:teuthology.orchestra.run.smithi183.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T13:05:46.620 INFO:teuthology.orchestra.run.smithi183.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T13:05:46.621 INFO:teuthology.run_tasks:Running task install... 2021-02-20T13:05:46.634 DEBUG:teuthology.task.install:project ceph 2021-02-20T13:05:46.635 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': '0d796331675b92762d821b40e70f6598bcea1178'}} 2021-02-20T13:05:46.635 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-mirror'], 'sha1': '0d796331675b92762d821b40e70f6598bcea1178'} 2021-02-20T13:05:46.635 INFO:teuthology.task.install:Using flavor: basic 2021-02-20T13:05:46.649 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2021-02-20T13:05:46.650 INFO:teuthology.task.install:extra packages: ['rbd-mirror'] 2021-02-20T13:05:46.650 DEBUG:teuthology.orchestra.run.smithi170:> sudo lsb_release -is 2021-02-20T13:05:46.696 INFO:teuthology.orchestra.run.smithi170.stdout:Ubuntu 2021-02-20T13:05:46.701 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-02-20T13:05:46.702 DEBUG:teuthology.orchestra.run.smithi183:> sudo lsb_release -is 2021-02-20T13:05:46.702 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-key list | grep Ceph 2021-02-20T13:05:46.755 INFO:teuthology.orchestra.run.smithi183.stdout:Ubuntu 2021-02-20T13:05:46.761 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-02-20T13:05:46.762 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-key list | grep Ceph 2021-02-20T13:05:46.792 INFO:teuthology.orchestra.run.smithi170.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-02-20T13:05:46.850 INFO:teuthology.orchestra.run.smithi170.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-02-20T13:05:46.850 INFO:teuthology.orchestra.run.smithi170.stdout:uid [ unknown] Ceph.com (release key) 2021-02-20T13:05:46.851 DEBUG:teuthology.orchestra.run.smithi170:> sudo lsb_release -is 2021-02-20T13:05:46.855 INFO:teuthology.orchestra.run.smithi183.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-02-20T13:05:46.902 INFO:teuthology.orchestra.run.smithi170.stdout:Ubuntu 2021-02-20T13:05:46.907 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-02-20T13:05:46.907 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-mirror on remote deb x86_64 2021-02-20T13:05:46.908 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=0d796331675b92762d821b40e70f6598bcea1178 2021-02-20T13:05:46.916 INFO:teuthology.orchestra.run.smithi183.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-02-20T13:05:46.916 INFO:teuthology.orchestra.run.smithi183.stdout:uid [ unknown] Ceph.com (release key) 2021-02-20T13:05:46.917 DEBUG:teuthology.orchestra.run.smithi183:> sudo lsb_release -is 2021-02-20T13:05:46.967 INFO:teuthology.orchestra.run.smithi183.stdout:Ubuntu 2021-02-20T13:05:46.972 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-02-20T13:05:46.973 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-mirror on remote deb x86_64 2021-02-20T13:05:46.973 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=0d796331675b92762d821b40e70f6598bcea1178 2021-02-20T13:05:47.080 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default/ 2021-02-20T13:05:47.080 INFO:teuthology.task.install.deb:Package version is 17.0.0-936-g0d796331-1bionic 2021-02-20T13:05:47.133 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default/ 2021-02-20T13:05:47.133 INFO:teuthology.task.install.deb:Package version is 17.0.0-936-g0d796331-1bionic 2021-02-20T13:05:47.382 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:05:47.383 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-02-20T13:05:47.398 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get update 2021-02-20T13:05:47.430 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:05:47.431 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-02-20T13:05:47.448 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get update 2021-02-20T13:05:47.624 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-02-20T13:05:47.698 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-02-20T13:05:47.699 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:2 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-02-20T13:05:47.741 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-02-20T13:05:47.766 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-02-20T13:05:47.819 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-02-20T13:05:47.826 INFO:teuthology.orchestra.run.smithi170.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic InRelease 2021-02-20T13:05:47.851 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:5 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-02-20T13:05:47.879 INFO:teuthology.orchestra.run.smithi183.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic InRelease 2021-02-20T13:05:47.897 INFO:teuthology.orchestra.run.smithi170.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2021-02-20T13:05:47.903 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:5 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-02-20T13:05:47.950 INFO:teuthology.orchestra.run.smithi183.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2021-02-20T13:05:47.970 INFO:teuthology.orchestra.run.smithi170.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic Release.gpg 2021-02-20T13:05:48.023 INFO:teuthology.orchestra.run.smithi183.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic Release.gpg 2021-02-20T13:05:48.043 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 Packages [16.6 kB] 2021-02-20T13:05:48.097 INFO:teuthology.orchestra.run.smithi183.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 Packages [16.6 kB] 2021-02-20T13:05:48.137 INFO:teuthology.orchestra.run.smithi170.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,115 B] 2021-02-20T13:05:48.201 INFO:teuthology.orchestra.run.smithi183.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,115 B] 2021-02-20T13:05:48.283 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 27.4 kB in 1s (44.0 kB/s) 2021-02-20T13:05:48.315 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 27.4 kB in 1s (38.3 kB/s) 2021-02-20T13:05:49.531 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2021-02-20T13:05:49.558 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2021-02-20T13:05:49.578 DEBUG:teuthology.orchestra.run.smithi183:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.0.0-936-g0d796331-1bionic cephadm=17.0.0-936-g0d796331-1bionic ceph-mds=17.0.0-936-g0d796331-1bionic ceph-mgr=17.0.0-936-g0d796331-1bionic ceph-common=17.0.0-936-g0d796331-1bionic ceph-fuse=17.0.0-936-g0d796331-1bionic ceph-test=17.0.0-936-g0d796331-1bionic radosgw=17.0.0-936-g0d796331-1bionic python3-rados=17.0.0-936-g0d796331-1bionic python3-rgw=17.0.0-936-g0d796331-1bionic python3-cephfs=17.0.0-936-g0d796331-1bionic python3-rbd=17.0.0-936-g0d796331-1bionic libcephfs2=17.0.0-936-g0d796331-1bionic libcephfs-dev=17.0.0-936-g0d796331-1bionic librados2=17.0.0-936-g0d796331-1bionic librbd1=17.0.0-936-g0d796331-1bionic rbd-fuse=17.0.0-936-g0d796331-1bionic rbd-mirror=17.0.0-936-g0d796331-1bionic 2021-02-20T13:05:49.599 DEBUG:teuthology.orchestra.run.smithi170:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.0.0-936-g0d796331-1bionic cephadm=17.0.0-936-g0d796331-1bionic ceph-mds=17.0.0-936-g0d796331-1bionic ceph-mgr=17.0.0-936-g0d796331-1bionic ceph-common=17.0.0-936-g0d796331-1bionic ceph-fuse=17.0.0-936-g0d796331-1bionic ceph-test=17.0.0-936-g0d796331-1bionic radosgw=17.0.0-936-g0d796331-1bionic python3-rados=17.0.0-936-g0d796331-1bionic python3-rgw=17.0.0-936-g0d796331-1bionic python3-cephfs=17.0.0-936-g0d796331-1bionic python3-rbd=17.0.0-936-g0d796331-1bionic libcephfs2=17.0.0-936-g0d796331-1bionic libcephfs-dev=17.0.0-936-g0d796331-1bionic librados2=17.0.0-936-g0d796331-1bionic librbd1=17.0.0-936-g0d796331-1bionic rbd-fuse=17.0.0-936-g0d796331-1bionic rbd-mirror=17.0.0-936-g0d796331-1bionic 2021-02-20T13:05:49.673 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2021-02-20T13:05:49.685 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2021-02-20T13:05:49.937 INFO:teuthology.orchestra.run.smithi170.stdout:Building dependency tree... 2021-02-20T13:05:49.938 INFO:teuthology.orchestra.run.smithi170.stdout:Reading state information... 2021-02-20T13:05:49.978 INFO:teuthology.orchestra.run.smithi183.stdout:Building dependency tree... 2021-02-20T13:05:49.979 INFO:teuthology.orchestra.run.smithi183.stdout:Reading state information... 2021-02-20T13:05:50.107 INFO:teuthology.orchestra.run.smithi170.stdout:The following additional packages will be installed: 2021-02-20T13:05:50.108 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2021-02-20T13:05:50.109 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd jq 2021-02-20T13:05:50.109 INFO:teuthology.orchestra.run.smithi170.stdout: libjaeger libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2021-02-20T13:05:50.109 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2021-02-20T13:05:50.110 INFO:teuthology.orchestra.run.smithi170.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2021-02-20T13:05:50.110 INFO:teuthology.orchestra.run.smithi170.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-02-20T13:05:50.110 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2021-02-20T13:05:50.110 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy3 python3-cryptography python3-cycler python3-dateutil 2021-02-20T13:05:50.111 INFO:teuthology.orchestra.run.smithi170.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2021-02-20T13:05:50.111 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2021-02-20T13:05:50.111 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-02-20T13:05:50.111 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2021-02-20T13:05:50.112 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2021-02-20T13:05:50.112 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2021-02-20T13:05:50.112 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2021-02-20T13:05:50.112 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2021-02-20T13:05:50.112 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2021-02-20T13:05:50.113 INFO:teuthology.orchestra.run.smithi170.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2021-02-20T13:05:50.113 INFO:teuthology.orchestra.run.smithi170.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2021-02-20T13:05:50.113 INFO:teuthology.orchestra.run.smithi170.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2021-02-20T13:05:50.113 INFO:teuthology.orchestra.run.smithi170.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2021-02-20T13:05:50.114 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2021-02-20T13:05:50.114 INFO:teuthology.orchestra.run.smithi170.stdout:Suggested packages: 2021-02-20T13:05:50.114 INFO:teuthology.orchestra.run.smithi170.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2021-02-20T13:05:50.115 INFO:teuthology.orchestra.run.smithi170.stdout: python-pytest-xdist python-mock python-cryptography-doc 2021-02-20T13:05:50.115 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2021-02-20T13:05:50.115 INFO:teuthology.orchestra.run.smithi170.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2021-02-20T13:05:50.115 INFO:teuthology.orchestra.run.smithi170.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2021-02-20T13:05:50.115 INFO:teuthology.orchestra.run.smithi170.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2021-02-20T13:05:50.116 INFO:teuthology.orchestra.run.smithi170.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2021-02-20T13:05:50.116 INFO:teuthology.orchestra.run.smithi170.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2021-02-20T13:05:50.116 INFO:teuthology.orchestra.run.smithi170.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2021-02-20T13:05:50.116 INFO:teuthology.orchestra.run.smithi170.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2021-02-20T13:05:50.117 INFO:teuthology.orchestra.run.smithi170.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2021-02-20T13:05:50.117 INFO:teuthology.orchestra.run.smithi170.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2021-02-20T13:05:50.117 INFO:teuthology.orchestra.run.smithi170.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2021-02-20T13:05:50.117 INFO:teuthology.orchestra.run.smithi170.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2021-02-20T13:05:50.117 INFO:teuthology.orchestra.run.smithi170.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2021-02-20T13:05:50.153 INFO:teuthology.orchestra.run.smithi183.stdout:The following additional packages will be installed: 2021-02-20T13:05:50.153 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2021-02-20T13:05:50.155 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd jq 2021-02-20T13:05:50.155 INFO:teuthology.orchestra.run.smithi183.stdout: libjaeger libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2021-02-20T13:05:50.155 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2021-02-20T13:05:50.155 INFO:teuthology.orchestra.run.smithi183.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2021-02-20T13:05:50.156 INFO:teuthology.orchestra.run.smithi183.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-02-20T13:05:50.156 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2021-02-20T13:05:50.156 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy3 python3-cryptography python3-cycler python3-dateutil 2021-02-20T13:05:50.156 INFO:teuthology.orchestra.run.smithi183.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2021-02-20T13:05:50.157 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2021-02-20T13:05:50.157 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-02-20T13:05:50.157 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2021-02-20T13:05:50.157 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2021-02-20T13:05:50.158 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2021-02-20T13:05:50.158 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2021-02-20T13:05:50.158 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2021-02-20T13:05:50.158 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2021-02-20T13:05:50.158 INFO:teuthology.orchestra.run.smithi183.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2021-02-20T13:05:50.159 INFO:teuthology.orchestra.run.smithi183.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2021-02-20T13:05:50.159 INFO:teuthology.orchestra.run.smithi183.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2021-02-20T13:05:50.159 INFO:teuthology.orchestra.run.smithi183.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2021-02-20T13:05:50.159 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2021-02-20T13:05:50.160 INFO:teuthology.orchestra.run.smithi183.stdout:Suggested packages: 2021-02-20T13:05:50.160 INFO:teuthology.orchestra.run.smithi183.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2021-02-20T13:05:50.160 INFO:teuthology.orchestra.run.smithi183.stdout: python-pytest-xdist python-mock python-cryptography-doc 2021-02-20T13:05:50.161 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2021-02-20T13:05:50.161 INFO:teuthology.orchestra.run.smithi183.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2021-02-20T13:05:50.161 INFO:teuthology.orchestra.run.smithi183.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2021-02-20T13:05:50.161 INFO:teuthology.orchestra.run.smithi183.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2021-02-20T13:05:50.162 INFO:teuthology.orchestra.run.smithi183.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2021-02-20T13:05:50.162 INFO:teuthology.orchestra.run.smithi183.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2021-02-20T13:05:50.162 INFO:teuthology.orchestra.run.smithi183.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2021-02-20T13:05:50.163 INFO:teuthology.orchestra.run.smithi183.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2021-02-20T13:05:50.163 INFO:teuthology.orchestra.run.smithi183.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2021-02-20T13:05:50.163 INFO:teuthology.orchestra.run.smithi183.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2021-02-20T13:05:50.163 INFO:teuthology.orchestra.run.smithi183.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2021-02-20T13:05:50.164 INFO:teuthology.orchestra.run.smithi183.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2021-02-20T13:05:50.164 INFO:teuthology.orchestra.run.smithi183.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2021-02-20T13:05:50.210 INFO:teuthology.orchestra.run.smithi170.stdout:The following NEW packages will be installed: 2021-02-20T13:05:50.210 INFO:teuthology.orchestra.run.smithi170.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-02-20T13:05:50.211 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2021-02-20T13:05:50.212 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm jq 2021-02-20T13:05:50.212 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-dev libcephfs2 libjaeger libjq1 libleveldb1v5 liblttng-ust0 2021-02-20T13:05:50.212 INFO:teuthology.orchestra.run.smithi170.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2021-02-20T13:05:50.213 INFO:teuthology.orchestra.run.smithi170.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2021-02-20T13:05:50.213 INFO:teuthology.orchestra.run.smithi170.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2021-02-20T13:05:50.213 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-02-20T13:05:50.213 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2021-02-20T13:05:50.213 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi-backend python3-cherrypy3 python3-cryptography python3-cycler 2021-02-20T13:05:50.214 INFO:teuthology.orchestra.run.smithi170.stdout: python3-dateutil python3-decorator python3-fasteners python3-html5lib 2021-02-20T13:05:50.214 INFO:teuthology.orchestra.run.smithi170.stdout: python3-httplib2 python3-jinja2 python3-joblib python3-json-pointer 2021-02-20T13:05:50.214 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch python3-jwt python3-kubernetes python3-logutils 2021-02-20T13:05:50.214 INFO:teuthology.orchestra.run.smithi170.stdout: python3-lxml python3-mako python3-markupsafe python3-matplotlib 2021-02-20T13:05:50.215 INFO:teuthology.orchestra.run.smithi170.stdout: python3-monotonic python3-numpy python3-oauth2client python3-olefile 2021-02-20T13:05:50.215 INFO:teuthology.orchestra.run.smithi170.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2021-02-20T13:05:50.215 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan python3-pil python3-pluggy python3-prettytable python3-py 2021-02-20T13:05:50.215 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyparsing 2021-02-20T13:05:50.216 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2021-02-20T13:05:50.216 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2021-02-20T13:05:50.216 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2021-02-20T13:05:50.216 INFO:teuthology.orchestra.run.smithi170.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2021-02-20T13:05:50.216 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2021-02-20T13:05:50.217 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2021-02-20T13:05:50.217 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug radosgw rbd-fuse rbd-mirror socat xmlstarlet 2021-02-20T13:05:50.218 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages will be upgraded: 2021-02-20T13:05:50.218 INFO:teuthology.orchestra.run.smithi170.stdout: librados2 librbd1 2021-02-20T13:05:50.264 INFO:teuthology.orchestra.run.smithi183.stdout:The following NEW packages will be installed: 2021-02-20T13:05:50.264 INFO:teuthology.orchestra.run.smithi183.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-02-20T13:05:50.265 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2021-02-20T13:05:50.265 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm jq 2021-02-20T13:05:50.267 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-dev libcephfs2 libjaeger libjq1 libleveldb1v5 liblttng-ust0 2021-02-20T13:05:50.268 INFO:teuthology.orchestra.run.smithi183.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2021-02-20T13:05:50.268 INFO:teuthology.orchestra.run.smithi183.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2021-02-20T13:05:50.269 INFO:teuthology.orchestra.run.smithi183.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2021-02-20T13:05:50.269 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-02-20T13:05:50.269 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2021-02-20T13:05:50.270 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi-backend python3-cherrypy3 python3-cryptography python3-cycler 2021-02-20T13:05:50.270 INFO:teuthology.orchestra.run.smithi183.stdout: python3-dateutil python3-decorator python3-fasteners python3-html5lib 2021-02-20T13:05:50.271 INFO:teuthology.orchestra.run.smithi183.stdout: python3-httplib2 python3-jinja2 python3-joblib python3-json-pointer 2021-02-20T13:05:50.271 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch python3-jwt python3-kubernetes python3-logutils 2021-02-20T13:05:50.272 INFO:teuthology.orchestra.run.smithi183.stdout: python3-lxml python3-mako python3-markupsafe python3-matplotlib 2021-02-20T13:05:50.272 INFO:teuthology.orchestra.run.smithi183.stdout: python3-monotonic python3-numpy python3-oauth2client python3-olefile 2021-02-20T13:05:50.272 INFO:teuthology.orchestra.run.smithi183.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2021-02-20T13:05:50.273 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan python3-pil python3-pluggy python3-prettytable python3-py 2021-02-20T13:05:50.273 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyparsing 2021-02-20T13:05:50.274 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2021-02-20T13:05:50.274 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2021-02-20T13:05:50.274 INFO:teuthology.orchestra.run.smithi183.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2021-02-20T13:05:50.275 INFO:teuthology.orchestra.run.smithi183.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2021-02-20T13:05:50.275 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2021-02-20T13:05:50.276 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2021-02-20T13:05:50.276 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug radosgw rbd-fuse rbd-mirror socat xmlstarlet 2021-02-20T13:05:50.276 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages will be upgraded: 2021-02-20T13:05:50.277 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 librbd1 2021-02-20T13:05:50.389 INFO:teuthology.orchestra.run.smithi170.stdout:2 upgraded, 112 newly installed, 0 to remove and 34 not upgraded. 2021-02-20T13:05:50.389 INFO:teuthology.orchestra.run.smithi170.stdout:Need to get 169 MB of archives. 2021-02-20T13:05:50.389 INFO:teuthology.orchestra.run.smithi170.stdout:After this operation, 793 MB of additional disk space will be used. 2021-02-20T13:05:50.390 INFO:teuthology.orchestra.run.smithi170.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 libjaeger amd64 17.0.0-936-g0d796331-1bionic [3,720 B] 2021-02-20T13:05:50.399 INFO:teuthology.orchestra.run.smithi170.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2021-02-20T13:05:50.457 INFO:teuthology.orchestra.run.smithi183.stdout:2 upgraded, 112 newly installed, 0 to remove and 34 not upgraded. 2021-02-20T13:05:50.457 INFO:teuthology.orchestra.run.smithi183.stdout:Need to get 169 MB of archives. 2021-02-20T13:05:50.457 INFO:teuthology.orchestra.run.smithi183.stdout:After this operation, 793 MB of additional disk space will be used. 2021-02-20T13:05:50.457 INFO:teuthology.orchestra.run.smithi183.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2021-02-20T13:05:50.473 INFO:teuthology.orchestra.run.smithi183.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 libjaeger amd64 17.0.0-936-g0d796331-1bionic [3,720 B] 2021-02-20T13:05:50.475 INFO:teuthology.orchestra.run.smithi170.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 17.0.0-936-g0d796331-1bionic [3,123 kB] 2021-02-20T13:05:50.478 INFO:teuthology.orchestra.run.smithi170.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2021-02-20T13:05:50.538 INFO:teuthology.orchestra.run.smithi183.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2021-02-20T13:05:50.566 INFO:teuthology.orchestra.run.smithi183.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 17.0.0-936-g0d796331-1bionic [3,123 kB] 2021-02-20T13:05:50.728 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2021-02-20T13:05:50.742 INFO:teuthology.orchestra.run.smithi170.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2021-02-20T13:05:50.753 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2021-02-20T13:05:50.790 INFO:teuthology.orchestra.run.smithi183.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2021-02-20T13:05:50.803 INFO:teuthology.orchestra.run.smithi183.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2021-02-20T13:05:50.821 INFO:teuthology.orchestra.run.smithi183.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2021-02-20T13:05:50.855 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2021-02-20T13:05:50.859 INFO:teuthology.orchestra.run.smithi170.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2021-02-20T13:05:50.886 INFO:teuthology.orchestra.run.smithi170.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2021-02-20T13:05:50.893 INFO:teuthology.orchestra.run.smithi170.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2021-02-20T13:05:50.900 INFO:teuthology.orchestra.run.smithi170.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2021-02-20T13:05:50.910 INFO:teuthology.orchestra.run.smithi170.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2021-02-20T13:05:50.916 INFO:teuthology.orchestra.run.smithi170.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2021-02-20T13:05:50.916 INFO:teuthology.orchestra.run.smithi183.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2021-02-20T13:05:50.921 INFO:teuthology.orchestra.run.smithi183.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2021-02-20T13:05:50.944 INFO:teuthology.orchestra.run.smithi170.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2021-02-20T13:05:50.949 INFO:teuthology.orchestra.run.smithi170.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2021-02-20T13:05:50.954 INFO:teuthology.orchestra.run.smithi183.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2021-02-20T13:05:50.961 INFO:teuthology.orchestra.run.smithi183.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2021-02-20T13:05:50.966 INFO:teuthology.orchestra.run.smithi170.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2021-02-20T13:05:50.969 INFO:teuthology.orchestra.run.smithi183.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2021-02-20T13:05:50.976 INFO:teuthology.orchestra.run.smithi170.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2021-02-20T13:05:50.978 INFO:teuthology.orchestra.run.smithi170.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2021-02-20T13:05:50.979 INFO:teuthology.orchestra.run.smithi183.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2021-02-20T13:05:50.988 INFO:teuthology.orchestra.run.smithi183.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2021-02-20T13:05:51.012 INFO:teuthology.orchestra.run.smithi183.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2021-02-20T13:05:51.015 INFO:teuthology.orchestra.run.smithi183.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2021-02-20T13:05:51.034 INFO:teuthology.orchestra.run.smithi183.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2021-02-20T13:05:51.044 INFO:teuthology.orchestra.run.smithi170.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2021-02-20T13:05:51.045 INFO:teuthology.orchestra.run.smithi170.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2021-02-20T13:05:51.047 INFO:teuthology.orchestra.run.smithi183.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2021-02-20T13:05:51.047 INFO:teuthology.orchestra.run.smithi170.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2021-02-20T13:05:51.048 INFO:teuthology.orchestra.run.smithi170.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2021-02-20T13:05:51.048 INFO:teuthology.orchestra.run.smithi170.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2021-02-20T13:05:51.052 INFO:teuthology.orchestra.run.smithi170.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2021-02-20T13:05:51.054 INFO:teuthology.orchestra.run.smithi170.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2021-02-20T13:05:51.114 INFO:teuthology.orchestra.run.smithi183.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2021-02-20T13:05:51.115 INFO:teuthology.orchestra.run.smithi183.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2021-02-20T13:05:51.115 INFO:teuthology.orchestra.run.smithi183.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2021-02-20T13:05:51.117 INFO:teuthology.orchestra.run.smithi183.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2021-02-20T13:05:51.118 INFO:teuthology.orchestra.run.smithi183.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2021-02-20T13:05:51.119 INFO:teuthology.orchestra.run.smithi183.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2021-02-20T13:05:51.121 INFO:teuthology.orchestra.run.smithi183.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2021-02-20T13:05:51.123 INFO:teuthology.orchestra.run.smithi170.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2021-02-20T13:05:51.124 INFO:teuthology.orchestra.run.smithi170.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2021-02-20T13:05:51.125 INFO:teuthology.orchestra.run.smithi183.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2021-02-20T13:05:51.144 INFO:teuthology.orchestra.run.smithi170.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2021-02-20T13:05:51.195 INFO:teuthology.orchestra.run.smithi183.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2021-02-20T13:05:51.196 INFO:teuthology.orchestra.run.smithi183.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2021-02-20T13:05:51.201 INFO:teuthology.orchestra.run.smithi170.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2021-02-20T13:05:51.202 INFO:teuthology.orchestra.run.smithi170.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2021-02-20T13:05:51.203 INFO:teuthology.orchestra.run.smithi170.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2021-02-20T13:05:51.206 INFO:teuthology.orchestra.run.smithi170.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2021-02-20T13:05:51.215 INFO:teuthology.orchestra.run.smithi170.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2021-02-20T13:05:51.218 INFO:teuthology.orchestra.run.smithi170.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2021-02-20T13:05:51.219 INFO:teuthology.orchestra.run.smithi170.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2021-02-20T13:05:51.275 INFO:teuthology.orchestra.run.smithi183.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2021-02-20T13:05:51.276 INFO:teuthology.orchestra.run.smithi183.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2021-02-20T13:05:51.277 INFO:teuthology.orchestra.run.smithi183.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2021-02-20T13:05:51.278 INFO:teuthology.orchestra.run.smithi183.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2021-02-20T13:05:51.279 INFO:teuthology.orchestra.run.smithi170.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2021-02-20T13:05:51.281 INFO:teuthology.orchestra.run.smithi183.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2021-02-20T13:05:51.289 INFO:teuthology.orchestra.run.smithi183.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2021-02-20T13:05:51.293 INFO:teuthology.orchestra.run.smithi183.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2021-02-20T13:05:51.294 INFO:teuthology.orchestra.run.smithi183.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2021-02-20T13:05:51.356 INFO:teuthology.orchestra.run.smithi183.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2021-02-20T13:05:51.364 INFO:teuthology.orchestra.run.smithi170.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2021-02-20T13:05:51.365 INFO:teuthology.orchestra.run.smithi170.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2021-02-20T13:05:51.392 INFO:teuthology.orchestra.run.smithi170.stdout:Get:40 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 17.0.0-936-g0d796331-1bionic [3,163 kB] 2021-02-20T13:05:51.445 INFO:teuthology.orchestra.run.smithi183.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2021-02-20T13:05:51.446 INFO:teuthology.orchestra.run.smithi183.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2021-02-20T13:05:51.494 INFO:teuthology.orchestra.run.smithi183.stdout:Get:40 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 17.0.0-936-g0d796331-1bionic [3,163 kB] 2021-02-20T13:05:51.634 INFO:teuthology.orchestra.run.smithi170.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2021-02-20T13:05:51.676 INFO:teuthology.orchestra.run.smithi170.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2021-02-20T13:05:51.678 INFO:teuthology.orchestra.run.smithi170.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2021-02-20T13:05:51.716 INFO:teuthology.orchestra.run.smithi170.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2021-02-20T13:05:51.717 INFO:teuthology.orchestra.run.smithi170.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2021-02-20T13:05:51.717 INFO:teuthology.orchestra.run.smithi170.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2021-02-20T13:05:51.718 INFO:teuthology.orchestra.run.smithi170.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2021-02-20T13:05:51.718 INFO:teuthology.orchestra.run.smithi170.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2021-02-20T13:05:51.719 INFO:teuthology.orchestra.run.smithi170.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2021-02-20T13:05:51.720 INFO:teuthology.orchestra.run.smithi170.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2021-02-20T13:05:51.734 INFO:teuthology.orchestra.run.smithi183.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2021-02-20T13:05:51.776 INFO:teuthology.orchestra.run.smithi183.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2021-02-20T13:05:51.777 INFO:teuthology.orchestra.run.smithi183.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2021-02-20T13:05:51.793 INFO:teuthology.orchestra.run.smithi170.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2021-02-20T13:05:51.803 INFO:teuthology.orchestra.run.smithi170.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2021-02-20T13:05:51.813 INFO:teuthology.orchestra.run.smithi183.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2021-02-20T13:05:51.814 INFO:teuthology.orchestra.run.smithi183.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2021-02-20T13:05:51.815 INFO:teuthology.orchestra.run.smithi183.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2021-02-20T13:05:51.816 INFO:teuthology.orchestra.run.smithi183.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2021-02-20T13:05:51.816 INFO:teuthology.orchestra.run.smithi183.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2021-02-20T13:05:51.816 INFO:teuthology.orchestra.run.smithi183.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2021-02-20T13:05:51.817 INFO:teuthology.orchestra.run.smithi183.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2021-02-20T13:05:51.869 INFO:teuthology.orchestra.run.smithi170.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2021-02-20T13:05:51.870 INFO:teuthology.orchestra.run.smithi170.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2021-02-20T13:05:51.877 INFO:teuthology.orchestra.run.smithi170.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2021-02-20T13:05:51.877 INFO:teuthology.orchestra.run.smithi170.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2021-02-20T13:05:51.878 INFO:teuthology.orchestra.run.smithi170.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2021-02-20T13:05:51.880 INFO:teuthology.orchestra.run.smithi170.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2021-02-20T13:05:51.881 INFO:teuthology.orchestra.run.smithi170.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2021-02-20T13:05:51.881 INFO:teuthology.orchestra.run.smithi170.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2021-02-20T13:05:51.896 INFO:teuthology.orchestra.run.smithi183.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2021-02-20T13:05:51.897 INFO:teuthology.orchestra.run.smithi183.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2021-02-20T13:05:51.901 INFO:teuthology.orchestra.run.smithi183.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2021-02-20T13:05:51.901 INFO:teuthology.orchestra.run.smithi183.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2021-02-20T13:05:51.904 INFO:teuthology.orchestra.run.smithi183.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2021-02-20T13:05:51.905 INFO:teuthology.orchestra.run.smithi183.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2021-02-20T13:05:51.947 INFO:teuthology.orchestra.run.smithi170.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2021-02-20T13:05:51.953 INFO:teuthology.orchestra.run.smithi170.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2021-02-20T13:05:51.977 INFO:teuthology.orchestra.run.smithi183.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2021-02-20T13:05:51.983 INFO:teuthology.orchestra.run.smithi183.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2021-02-20T13:05:51.989 INFO:teuthology.orchestra.run.smithi183.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2021-02-20T13:05:51.989 INFO:teuthology.orchestra.run.smithi183.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2021-02-20T13:05:51.993 INFO:teuthology.orchestra.run.smithi170.stdout:Get:63 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 17.0.0-936-g0d796331-1bionic [664 kB] 2021-02-20T13:05:52.025 INFO:teuthology.orchestra.run.smithi170.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2021-02-20T13:05:52.033 INFO:teuthology.orchestra.run.smithi170.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2021-02-20T13:05:52.035 INFO:teuthology.orchestra.run.smithi170.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2021-02-20T13:05:52.037 INFO:teuthology.orchestra.run.smithi170.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2021-02-20T13:05:52.039 INFO:teuthology.orchestra.run.smithi170.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2021-02-20T13:05:52.042 INFO:teuthology.orchestra.run.smithi170.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2021-02-20T13:05:52.044 INFO:teuthology.orchestra.run.smithi170.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2021-02-20T13:05:52.045 INFO:teuthology.orchestra.run.smithi170.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2021-02-20T13:05:52.059 INFO:teuthology.orchestra.run.smithi183.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2021-02-20T13:05:52.064 INFO:teuthology.orchestra.run.smithi183.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2021-02-20T13:05:52.071 INFO:teuthology.orchestra.run.smithi183.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2021-02-20T13:05:52.074 INFO:teuthology.orchestra.run.smithi183.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2021-02-20T13:05:52.074 INFO:teuthology.orchestra.run.smithi183.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2021-02-20T13:05:52.075 INFO:teuthology.orchestra.run.smithi183.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2021-02-20T13:05:52.084 INFO:teuthology.orchestra.run.smithi183.stdout:Get:67 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 17.0.0-936-g0d796331-1bionic [664 kB] 2021-02-20T13:05:52.092 INFO:teuthology.orchestra.run.smithi170.stdout:Get:72 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 17.0.0-936-g0d796331-1bionic [339 kB] 2021-02-20T13:05:52.103 INFO:teuthology.orchestra.run.smithi170.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2021-02-20T13:05:52.104 INFO:teuthology.orchestra.run.smithi170.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2021-02-20T13:05:52.138 INFO:teuthology.orchestra.run.smithi183.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2021-02-20T13:05:52.143 INFO:teuthology.orchestra.run.smithi183.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2021-02-20T13:05:52.144 INFO:teuthology.orchestra.run.smithi170.stdout:Get:75 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 17.0.0-936-g0d796331-1bionic [21.7 kB] 2021-02-20T13:05:52.145 INFO:teuthology.orchestra.run.smithi170.stdout:Get:76 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 17.0.0-936-g0d796331-1bionic [163 kB] 2021-02-20T13:05:52.147 INFO:teuthology.orchestra.run.smithi183.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2021-02-20T13:05:52.147 INFO:teuthology.orchestra.run.smithi183.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2021-02-20T13:05:52.151 INFO:teuthology.orchestra.run.smithi170.stdout:Get:77 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 17.0.0-936-g0d796331-1bionic [29.3 kB] 2021-02-20T13:05:52.154 INFO:teuthology.orchestra.run.smithi170.stdout:Get:78 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 17.0.0-936-g0d796331-1bionic [336 kB] 2021-02-20T13:05:52.183 INFO:teuthology.orchestra.run.smithi170.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.3 [1,097 kB] 2021-02-20T13:05:52.187 INFO:teuthology.orchestra.run.smithi183.stdout:Get:72 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 17.0.0-936-g0d796331-1bionic [339 kB] 2021-02-20T13:05:52.204 INFO:teuthology.orchestra.run.smithi170.stdout:Get:80 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 17.0.0-936-g0d796331-1bionic [3,294 kB] 2021-02-20T13:05:52.208 INFO:teuthology.orchestra.run.smithi170.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2021-02-20T13:05:52.209 INFO:teuthology.orchestra.run.smithi170.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2021-02-20T13:05:52.209 INFO:teuthology.orchestra.run.smithi170.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2021-02-20T13:05:52.219 INFO:teuthology.orchestra.run.smithi183.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2021-02-20T13:05:52.220 INFO:teuthology.orchestra.run.smithi183.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2021-02-20T13:05:52.222 INFO:teuthology.orchestra.run.smithi183.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.3 [1,097 kB] 2021-02-20T13:05:52.233 INFO:teuthology.orchestra.run.smithi183.stdout:Get:76 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 17.0.0-936-g0d796331-1bionic [21.7 kB] 2021-02-20T13:05:52.235 INFO:teuthology.orchestra.run.smithi183.stdout:Get:77 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 17.0.0-936-g0d796331-1bionic [163 kB] 2021-02-20T13:05:52.246 INFO:teuthology.orchestra.run.smithi183.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2021-02-20T13:05:52.247 INFO:teuthology.orchestra.run.smithi183.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2021-02-20T13:05:52.247 INFO:teuthology.orchestra.run.smithi183.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2021-02-20T13:05:52.250 INFO:teuthology.orchestra.run.smithi183.stdout:Get:81 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 17.0.0-936-g0d796331-1bionic [29.3 kB] 2021-02-20T13:05:52.257 INFO:teuthology.orchestra.run.smithi183.stdout:Get:82 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 17.0.0-936-g0d796331-1bionic [336 kB] 2021-02-20T13:05:52.304 INFO:teuthology.orchestra.run.smithi183.stdout:Get:83 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 17.0.0-936-g0d796331-1bionic [3,294 kB] 2021-02-20T13:05:52.352 INFO:teuthology.orchestra.run.smithi170.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2021-02-20T13:05:52.353 INFO:teuthology.orchestra.run.smithi170.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2021-02-20T13:05:52.357 INFO:teuthology.orchestra.run.smithi170.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.4 [329 kB] 2021-02-20T13:05:52.364 INFO:teuthology.orchestra.run.smithi170.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2021-02-20T13:05:52.364 INFO:teuthology.orchestra.run.smithi170.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2021-02-20T13:05:52.364 INFO:teuthology.orchestra.run.smithi170.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2021-02-20T13:05:52.365 INFO:teuthology.orchestra.run.smithi170.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2021-02-20T13:05:52.365 INFO:teuthology.orchestra.run.smithi170.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2021-02-20T13:05:52.365 INFO:teuthology.orchestra.run.smithi170.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2021-02-20T13:05:52.385 INFO:teuthology.orchestra.run.smithi183.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2021-02-20T13:05:52.386 INFO:teuthology.orchestra.run.smithi183.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2021-02-20T13:05:52.387 INFO:teuthology.orchestra.run.smithi183.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.4 [329 kB] 2021-02-20T13:05:52.393 INFO:teuthology.orchestra.run.smithi183.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2021-02-20T13:05:52.393 INFO:teuthology.orchestra.run.smithi183.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2021-02-20T13:05:52.394 INFO:teuthology.orchestra.run.smithi183.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2021-02-20T13:05:52.395 INFO:teuthology.orchestra.run.smithi183.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2021-02-20T13:05:52.395 INFO:teuthology.orchestra.run.smithi183.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2021-02-20T13:05:52.396 INFO:teuthology.orchestra.run.smithi183.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2021-02-20T13:05:52.581 INFO:teuthology.orchestra.run.smithi170.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 17.0.0-936-g0d796331-1bionic [99.3 kB] 2021-02-20T13:05:52.587 INFO:teuthology.orchestra.run.smithi170.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 17.0.0-936-g0d796331-1bionic [414 kB] 2021-02-20T13:05:52.646 INFO:teuthology.orchestra.run.smithi170.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 17.0.0-936-g0d796331-1bionic [21.2 MB] 2021-02-20T13:05:52.694 INFO:teuthology.orchestra.run.smithi183.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 17.0.0-936-g0d796331-1bionic [99.3 kB] 2021-02-20T13:05:52.698 INFO:teuthology.orchestra.run.smithi183.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 17.0.0-936-g0d796331-1bionic [414 kB] 2021-02-20T13:05:52.746 INFO:teuthology.orchestra.run.smithi183.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 17.0.0-936-g0d796331-1bionic [21.2 MB] 2021-02-20T13:05:54.344 INFO:teuthology.orchestra.run.smithi170.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 17.0.0-936-g0d796331-1bionic [5,633 kB] 2021-02-20T13:05:54.575 INFO:teuthology.orchestra.run.smithi183.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 17.0.0-936-g0d796331-1bionic [5,633 kB] 2021-02-20T13:05:54.791 INFO:teuthology.orchestra.run.smithi170.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 17.0.0-936-g0d796331-1bionic [185 kB] 2021-02-20T13:05:54.812 INFO:teuthology.orchestra.run.smithi170.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 17.0.0-936-g0d796331-1bionic [1,366 kB] 2021-02-20T13:05:54.901 INFO:teuthology.orchestra.run.smithi170.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 17.0.0-936-g0d796331-1bionic [6,660 kB] 2021-02-20T13:05:55.021 INFO:teuthology.orchestra.run.smithi183.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 17.0.0-936-g0d796331-1bionic [185 kB] 2021-02-20T13:05:55.054 INFO:teuthology.orchestra.run.smithi183.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 17.0.0-936-g0d796331-1bionic [1,366 kB] 2021-02-20T13:05:55.159 INFO:teuthology.orchestra.run.smithi183.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 17.0.0-936-g0d796331-1bionic [6,660 kB] 2021-02-20T13:05:55.419 INFO:teuthology.orchestra.run.smithi170.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 17.0.0-936-g0d796331-1bionic [24.9 MB] 2021-02-20T13:05:55.699 INFO:teuthology.orchestra.run.smithi183.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 17.0.0-936-g0d796331-1bionic [24.9 MB] 2021-02-20T13:05:57.581 INFO:teuthology.orchestra.run.smithi170.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 17.0.0-936-g0d796331-1bionic [3,808 B] 2021-02-20T13:05:57.581 INFO:teuthology.orchestra.run.smithi170.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 17.0.0-936-g0d796331-1bionic [770 kB] 2021-02-20T13:05:57.654 INFO:teuthology.orchestra.run.smithi170.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 17.0.0-936-g0d796331-1bionic [2,152 kB] 2021-02-20T13:05:57.809 INFO:teuthology.orchestra.run.smithi183.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 17.0.0-936-g0d796331-1bionic [3,808 B] 2021-02-20T13:05:57.810 INFO:teuthology.orchestra.run.smithi183.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 17.0.0-936-g0d796331-1bionic [770 kB] 2021-02-20T13:05:57.881 INFO:teuthology.orchestra.run.smithi183.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 17.0.0-936-g0d796331-1bionic [2,152 kB] 2021-02-20T13:05:57.940 INFO:teuthology.orchestra.run.smithi170.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 17.0.0-936-g0d796331-1bionic [62.2 kB] 2021-02-20T13:05:57.942 INFO:teuthology.orchestra.run.smithi170.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 17.0.0-936-g0d796331-1bionic [67.9 kB] 2021-02-20T13:05:57.945 INFO:teuthology.orchestra.run.smithi170.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 17.0.0-936-g0d796331-1bionic [2,328 kB] 2021-02-20T13:05:58.068 INFO:teuthology.orchestra.run.smithi183.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 17.0.0-936-g0d796331-1bionic [62.2 kB] 2021-02-20T13:05:58.096 INFO:teuthology.orchestra.run.smithi183.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 17.0.0-936-g0d796331-1bionic [67.9 kB] 2021-02-20T13:05:58.099 INFO:teuthology.orchestra.run.smithi183.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 17.0.0-936-g0d796331-1bionic [2,328 kB] 2021-02-20T13:05:58.232 INFO:teuthology.orchestra.run.smithi170.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 17.0.0-936-g0d796331-1bionic [7,896 kB] 2021-02-20T13:05:58.315 INFO:teuthology.orchestra.run.smithi183.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 17.0.0-936-g0d796331-1bionic [7,896 kB] 2021-02-20T13:05:59.064 INFO:teuthology.orchestra.run.smithi183.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 17.0.0-936-g0d796331-1bionic [3,924 B] 2021-02-20T13:05:59.065 INFO:teuthology.orchestra.run.smithi183.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 17.0.0-936-g0d796331-1bionic [23.3 kB] 2021-02-20T13:05:59.065 INFO:teuthology.orchestra.run.smithi183.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 17.0.0-936-g0d796331-1bionic [45.0 MB] 2021-02-20T13:05:59.224 INFO:teuthology.orchestra.run.smithi170.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 17.0.0-936-g0d796331-1bionic [3,924 B] 2021-02-20T13:05:59.224 INFO:teuthology.orchestra.run.smithi170.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 17.0.0-936-g0d796331-1bionic [23.3 kB] 2021-02-20T13:05:59.225 INFO:teuthology.orchestra.run.smithi170.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 17.0.0-936-g0d796331-1bionic [45.0 MB] 2021-02-20T13:06:03.189 INFO:teuthology.orchestra.run.smithi183.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 17.0.0-936-g0d796331-1bionic [19.8 kB] 2021-02-20T13:06:03.190 INFO:teuthology.orchestra.run.smithi183.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 17.0.0-936-g0d796331-1bionic [10.1 MB] 2021-02-20T13:06:04.307 INFO:teuthology.orchestra.run.smithi183.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 17.0.0-936-g0d796331-1bionic [77.6 kB] 2021-02-20T13:06:04.309 INFO:teuthology.orchestra.run.smithi183.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 rbd-mirror amd64 17.0.0-936-g0d796331-1bionic [3,330 kB] 2021-02-20T13:06:05.199 INFO:teuthology.orchestra.run.smithi170.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 17.0.0-936-g0d796331-1bionic [19.8 kB] 2021-02-20T13:06:05.200 INFO:teuthology.orchestra.run.smithi170.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 17.0.0-936-g0d796331-1bionic [10.1 MB] 2021-02-20T13:06:05.463 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 169 MB in 14s (11.7 MB/s) 2021-02-20T13:06:05.606 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-uritemplate. 2021-02-20T13:06:05.653 INFO:teuthology.orchestra.run.smithi183.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137611 files and directories currently installed.) 2021-02-20T13:06:05.657 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../000-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2021-02-20T13:06:05.704 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2021-02-20T13:06:05.914 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libjaeger. 2021-02-20T13:06:05.929 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../001-libjaeger_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:05.964 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libjaeger (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:06.234 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-02-20T13:06:06.247 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../002-liblttng-ust0_2.10.1-1_amd64.deb ... 2021-02-20T13:06:06.317 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2021-02-20T13:06:06.411 INFO:teuthology.orchestra.run.smithi170.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 17.0.0-936-g0d796331-1bionic [77.6 kB] 2021-02-20T13:06:06.424 INFO:teuthology.orchestra.run.smithi170.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-mgolub-testing/0d796331675b92762d821b40e70f6598bcea1178/ubuntu/bionic/flavors/default bionic/main amd64 rbd-mirror amd64 17.0.0-936-g0d796331-1bionic [3,330 kB] 2021-02-20T13:06:06.602 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../003-librbd1_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:06.712 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librbd1 (17.0.0-936-g0d796331-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-02-20T13:06:07.363 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../004-librados2_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:07.475 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librados2 (17.0.0-936-g0d796331-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-02-20T13:06:07.526 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 169 MB in 17s (10.2 MB/s) 2021-02-20T13:06:07.696 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-uritemplate. 2021-02-20T13:06:07.747 INFO:teuthology.orchestra.run.smithi170.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137611 files and directories currently installed.) 2021-02-20T13:06:07.750 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../000-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2021-02-20T13:06:07.795 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2021-02-20T13:06:08.039 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libjaeger. 2021-02-20T13:06:08.053 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../001-libjaeger_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:08.061 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libcephfs2. 2021-02-20T13:06:08.078 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../005-libcephfs2_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:08.097 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libjaeger (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:08.112 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libcephfs2 (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:08.366 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-02-20T13:06:08.380 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../002-liblttng-ust0_2.10.1-1_amd64.deb ... 2021-02-20T13:06:08.414 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-rados. 2021-02-20T13:06:08.430 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../006-python3-rados_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:08.450 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2021-02-20T13:06:08.472 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-rados (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:08.777 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-02-20T13:06:08.778 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../003-librbd1_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:08.790 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../007-python3-ceph-argparse_17.0.0-936-g0d796331-1bionic_all.deb ... 2021-02-20T13:06:08.827 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-ceph-argparse (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:08.879 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librbd1 (17.0.0-936-g0d796331-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-02-20T13:06:09.054 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cephfs. 2021-02-20T13:06:09.069 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../008-python3-cephfs_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:09.104 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cephfs (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:09.432 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-ceph-common. 2021-02-20T13:06:09.444 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../009-python3-ceph-common_17.0.0-936-g0d796331-1bionic_all.deb ... 2021-02-20T13:06:09.483 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-ceph-common (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:09.539 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../004-librados2_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:09.683 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librados2 (17.0.0-936-g0d796331-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-02-20T13:06:09.802 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-prettytable. 2021-02-20T13:06:09.815 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../010-python3-prettytable_0.7.2-3_all.deb ... 2021-02-20T13:06:09.853 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2021-02-20T13:06:10.087 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-rbd. 2021-02-20T13:06:10.100 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../011-python3-rbd_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:10.129 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-rbd (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:10.237 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libcephfs2. 2021-02-20T13:06:10.250 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../005-libcephfs2_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:10.279 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libcephfs2 (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:10.440 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-02-20T13:06:10.456 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../012-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2021-02-20T13:06:10.498 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-02-20T13:06:10.623 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rados. 2021-02-20T13:06:10.637 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../006-python3-rados_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:10.681 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rados (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:10.834 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-02-20T13:06:10.848 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../013-librdkafka1_0.11.3-1build1_amd64.deb ... 2021-02-20T13:06:10.884 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2021-02-20T13:06:11.000 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-02-20T13:06:11.015 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../007-python3-ceph-argparse_17.0.0-936-g0d796331-1bionic_all.deb ... 2021-02-20T13:06:11.050 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-ceph-argparse (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:11.220 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package librgw2. 2021-02-20T13:06:11.234 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../014-librgw2_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:11.253 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cephfs. 2021-02-20T13:06:11.262 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librgw2 (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:11.267 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../008-python3-cephfs_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:11.303 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cephfs (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:11.605 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-ceph-common. 2021-02-20T13:06:11.624 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../009-python3-ceph-common_17.0.0-936-g0d796331-1bionic_all.deb ... 2021-02-20T13:06:11.672 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-ceph-common (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:11.781 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-rgw. 2021-02-20T13:06:11.794 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../015-python3-rgw_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:11.827 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-rgw (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:11.983 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-prettytable. 2021-02-20T13:06:11.997 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../010-python3-prettytable_0.7.2-3_all.deb ... 2021-02-20T13:06:12.033 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2021-02-20T13:06:12.125 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2021-02-20T13:06:12.140 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../016-libsnappy1v5_1.1.7-1_amd64.deb ... 2021-02-20T13:06:12.192 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2021-02-20T13:06:12.269 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rbd. 2021-02-20T13:06:12.285 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../011-python3-rbd_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:12.319 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rbd (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:12.519 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2021-02-20T13:06:12.534 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../017-libleveldb1v5_1.20-2_amd64.deb ... 2021-02-20T13:06:12.578 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2021-02-20T13:06:12.663 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-02-20T13:06:12.680 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../012-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2021-02-20T13:06:12.714 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-02-20T13:06:12.864 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package liboath0. 2021-02-20T13:06:12.877 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../018-liboath0_2.6.1-1_amd64.deb ... 2021-02-20T13:06:12.906 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking liboath0 (2.6.1-1) ... 2021-02-20T13:06:13.033 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-02-20T13:06:13.047 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../013-librdkafka1_0.11.3-1build1_amd64.deb ... 2021-02-20T13:06:13.091 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2021-02-20T13:06:13.200 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libradosstriper1. 2021-02-20T13:06:13.217 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../019-libradosstriper1_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:13.259 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libradosstriper1 (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:13.575 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librgw2. 2021-02-20T13:06:13.575 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../014-librgw2_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:13.575 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librgw2 (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:13.737 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-common. 2021-02-20T13:06:13.749 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../020-ceph-common_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:13.786 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-common (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:14.014 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rgw. 2021-02-20T13:06:14.026 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../015-python3-rgw_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:14.055 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rgw (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:14.383 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2021-02-20T13:06:14.399 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../016-libsnappy1v5_1.1.7-1_amd64.deb ... 2021-02-20T13:06:14.441 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2021-02-20T13:06:14.744 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2021-02-20T13:06:14.757 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../017-libleveldb1v5_1.20-2_amd64.deb ... 2021-02-20T13:06:14.794 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2021-02-20T13:06:15.105 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package liboath0. 2021-02-20T13:06:15.121 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../018-liboath0_2.6.1-1_amd64.deb ... 2021-02-20T13:06:15.155 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking liboath0 (2.6.1-1) ... 2021-02-20T13:06:15.473 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libradosstriper1. 2021-02-20T13:06:15.474 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../019-libradosstriper1_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:15.501 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libradosstriper1 (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:15.795 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-base. 2021-02-20T13:06:15.799 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../021-ceph-base_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:15.859 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-base (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:15.952 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-common. 2021-02-20T13:06:15.970 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../020-ceph-common_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:16.011 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-common (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:16.763 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-dateutil. 2021-02-20T13:06:16.779 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../022-python3-dateutil_2.6.1-1_all.deb ... 2021-02-20T13:06:16.821 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2021-02-20T13:06:17.133 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-asn1crypto. 2021-02-20T13:06:17.148 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../023-python3-asn1crypto_0.24.0-1_all.deb ... 2021-02-20T13:06:17.191 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2021-02-20T13:06:17.452 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cffi-backend. 2021-02-20T13:06:17.468 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../024-python3-cffi-backend_1.11.5-1_amd64.deb ... 2021-02-20T13:06:17.510 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2021-02-20T13:06:17.846 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cryptography. 2021-02-20T13:06:17.859 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../025-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2021-02-20T13:06:17.896 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-02-20T13:06:17.933 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-base. 2021-02-20T13:06:17.946 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../021-ceph-base_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:17.992 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-base (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:18.266 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-openssl. 2021-02-20T13:06:18.281 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../026-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2021-02-20T13:06:18.324 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2021-02-20T13:06:18.594 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-02-20T13:06:18.608 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../027-ceph-mgr-modules-core_17.0.0-936-g0d796331-1bionic_all.deb ... 2021-02-20T13:06:18.652 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-modules-core (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:18.955 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-dateutil. 2021-02-20T13:06:18.968 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../022-python3-dateutil_2.6.1-1_all.deb ... 2021-02-20T13:06:19.026 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-bcrypt. 2021-02-20T13:06:19.038 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2021-02-20T13:06:19.039 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../028-python3-bcrypt_3.1.4-2_amd64.deb ... 2021-02-20T13:06:19.082 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2021-02-20T13:06:19.358 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-asn1crypto. 2021-02-20T13:06:19.371 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../023-python3-asn1crypto_0.24.0-1_all.deb ... 2021-02-20T13:06:19.384 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cherrypy3. 2021-02-20T13:06:19.398 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../029-python3-cherrypy3_8.9.1-2_all.deb ... 2021-02-20T13:06:19.415 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2021-02-20T13:06:19.443 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2021-02-20T13:06:19.694 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cffi-backend. 2021-02-20T13:06:19.707 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../024-python3-cffi-backend_1.11.5-1_amd64.deb ... 2021-02-20T13:06:19.752 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2021-02-20T13:06:19.771 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-jwt. 2021-02-20T13:06:19.786 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../030-python3-jwt_1.5.3+ds1-1_all.deb ... 2021-02-20T13:06:19.829 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2021-02-20T13:06:20.087 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cryptography. 2021-02-20T13:06:20.102 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../025-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2021-02-20T13:06:20.146 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-02-20T13:06:20.174 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-logutils. 2021-02-20T13:06:20.191 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../031-python3-logutils_0.3.3-5_all.deb ... 2021-02-20T13:06:20.224 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-logutils (0.3.3-5) ... 2021-02-20T13:06:20.569 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-markupsafe. 2021-02-20T13:06:20.582 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../032-python3-markupsafe_1.0-1build1_amd64.deb ... 2021-02-20T13:06:20.627 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2021-02-20T13:06:21.325 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-openssl. 2021-02-20T13:06:21.341 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../026-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2021-02-20T13:06:21.383 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2021-02-20T13:06:21.644 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-02-20T13:06:21.657 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../027-ceph-mgr-modules-core_17.0.0-936-g0d796331-1bionic_all.deb ... 2021-02-20T13:06:21.694 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-modules-core (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:21.763 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-mako. 2021-02-20T13:06:21.779 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../033-python3-mako_1.0.7+ds1-1_all.deb ... 2021-02-20T13:06:21.821 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2021-02-20T13:06:22.090 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-bcrypt. 2021-02-20T13:06:22.104 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../028-python3-bcrypt_3.1.4-2_amd64.deb ... 2021-02-20T13:06:22.141 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-simplegeneric. 2021-02-20T13:06:22.148 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2021-02-20T13:06:22.154 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../034-python3-simplegeneric_0.8.1-1_all.deb ... 2021-02-20T13:06:22.195 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2021-02-20T13:06:22.451 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cherrypy3. 2021-02-20T13:06:22.464 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../029-python3-cherrypy3_8.9.1-2_all.deb ... 2021-02-20T13:06:22.501 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2021-02-20T13:06:22.523 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-singledispatch. 2021-02-20T13:06:22.537 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../035-python3-singledispatch_3.4.0.3-2_all.deb ... 2021-02-20T13:06:22.583 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2021-02-20T13:06:22.829 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jwt. 2021-02-20T13:06:22.843 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../030-python3-jwt_1.5.3+ds1-1_all.deb ... 2021-02-20T13:06:22.888 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2021-02-20T13:06:22.903 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-webob. 2021-02-20T13:06:22.917 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../036-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2021-02-20T13:06:22.969 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2021-02-20T13:06:23.217 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-logutils. 2021-02-20T13:06:23.231 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../031-python3-logutils_0.3.3-5_all.deb ... 2021-02-20T13:06:23.274 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-logutils (0.3.3-5) ... 2021-02-20T13:06:23.289 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-bs4. 2021-02-20T13:06:23.302 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../037-python3-bs4_4.6.0-1_all.deb ... 2021-02-20T13:06:23.380 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2021-02-20T13:06:23.594 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-markupsafe. 2021-02-20T13:06:23.609 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../032-python3-markupsafe_1.0-1build1_amd64.deb ... 2021-02-20T13:06:23.660 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2021-02-20T13:06:23.700 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-waitress. 2021-02-20T13:06:23.715 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../038-python3-waitress_1.0.1-1_all.deb ... 2021-02-20T13:06:23.759 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-waitress (1.0.1-1) ... 2021-02-20T13:06:23.980 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-mako. 2021-02-20T13:06:23.996 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../033-python3-mako_1.0.7+ds1-1_all.deb ... 2021-02-20T13:06:24.038 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2021-02-20T13:06:24.061 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-tempita. 2021-02-20T13:06:24.075 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-2_all.deb ... 2021-02-20T13:06:24.123 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-tempita (0.5.2-2) ... 2021-02-20T13:06:24.358 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-simplegeneric. 2021-02-20T13:06:24.372 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../034-python3-simplegeneric_0.8.1-1_all.deb ... 2021-02-20T13:06:24.414 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-paste. 2021-02-20T13:06:24.416 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2021-02-20T13:06:24.427 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../040-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2021-02-20T13:06:24.489 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-02-20T13:06:24.737 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-singledispatch. 2021-02-20T13:06:24.753 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../035-python3-singledispatch_3.4.0.3-2_all.deb ... 2021-02-20T13:06:24.809 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-02-20T13:06:24.887 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2021-02-20T13:06:24.887 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2021-02-20T13:06:24.888 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2021-02-20T13:06:25.124 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-webob. 2021-02-20T13:06:25.140 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../036-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2021-02-20T13:06:25.145 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pastedeploy. 2021-02-20T13:06:25.160 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../042-python3-pastedeploy_1.5.2-4_all.deb ... 2021-02-20T13:06:25.183 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2021-02-20T13:06:25.203 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2021-02-20T13:06:25.510 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-bs4. 2021-02-20T13:06:25.522 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-webtest. 2021-02-20T13:06:25.524 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../037-python3-bs4_4.6.0-1_all.deb ... 2021-02-20T13:06:25.537 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../043-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2021-02-20T13:06:25.569 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2021-02-20T13:06:25.597 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2021-02-20T13:06:25.897 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-waitress. 2021-02-20T13:06:25.911 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../038-python3-waitress_1.0.1-1_all.deb ... 2021-02-20T13:06:25.925 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pecan. 2021-02-20T13:06:25.939 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../044-python3-pecan_1.2.1-2_all.deb ... 2021-02-20T13:06:25.955 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-waitress (1.0.1-1) ... 2021-02-20T13:06:25.984 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pecan (1.2.1-2) ... 2021-02-20T13:06:26.292 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-tempita. 2021-02-20T13:06:26.307 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-2_all.deb ... 2021-02-20T13:06:26.364 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-tempita (0.5.2-2) ... 2021-02-20T13:06:26.389 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-werkzeug. 2021-02-20T13:06:26.446 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../045-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2021-02-20T13:06:26.446 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-02-20T13:06:26.638 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-paste. 2021-02-20T13:06:26.653 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../040-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2021-02-20T13:06:26.695 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-02-20T13:06:26.825 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr. 2021-02-20T13:06:26.838 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../046-ceph-mgr_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:26.884 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:27.039 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-02-20T13:06:27.055 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2021-02-20T13:06:27.106 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2021-02-20T13:06:27.337 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mon. 2021-02-20T13:06:27.353 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../047-ceph-mon_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:27.395 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mon (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:27.410 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pastedeploy. 2021-02-20T13:06:27.425 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../042-python3-pastedeploy_1.5.2-4_all.deb ... 2021-02-20T13:06:27.479 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2021-02-20T13:06:27.770 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-webtest. 2021-02-20T13:06:27.787 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../043-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2021-02-20T13:06:27.828 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2021-02-20T13:06:28.165 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pecan. 2021-02-20T13:06:28.180 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../044-python3-pecan_1.2.1-2_all.deb ... 2021-02-20T13:06:28.223 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pecan (1.2.1-2) ... 2021-02-20T13:06:28.366 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-osd. 2021-02-20T13:06:28.381 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../048-ceph-osd_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:28.425 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-osd (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:28.587 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-werkzeug. 2021-02-20T13:06:28.603 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../045-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2021-02-20T13:06:28.661 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-02-20T13:06:29.031 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr. 2021-02-20T13:06:29.047 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../046-ceph-mgr_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:29.089 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:29.585 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mon. 2021-02-20T13:06:29.599 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../047-ceph-mon_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:29.653 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mon (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:30.532 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph. 2021-02-20T13:06:30.546 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../049-ceph_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:30.589 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:30.658 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-osd. 2021-02-20T13:06:30.671 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../048-ceph-osd_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:30.716 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-osd (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:30.942 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-fuse. 2021-02-20T13:06:30.957 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../050-ceph-fuse_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:31.009 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-fuse (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:31.429 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mds. 2021-02-20T13:06:31.444 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../051-ceph-mds_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:31.487 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mds (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:31.982 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package cephadm. 2021-02-20T13:06:31.995 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../052-cephadm_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:32.031 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking cephadm (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:32.380 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-jinja2. 2021-02-20T13:06:32.394 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../053-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2021-02-20T13:06:32.439 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-02-20T13:06:32.750 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-02-20T13:06:32.765 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../054-ceph-mgr-cephadm_17.0.0-936-g0d796331-1bionic_all.deb ... 2021-02-20T13:06:32.805 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph. 2021-02-20T13:06:32.808 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-cephadm (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:32.819 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../049-ceph_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:32.855 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:33.130 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-repoze.lru. 2021-02-20T13:06:33.143 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../055-python3-repoze.lru_0.7-1_all.deb ... 2021-02-20T13:06:33.179 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2021-02-20T13:06:33.191 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-fuse. 2021-02-20T13:06:33.206 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../050-ceph-fuse_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:33.249 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-fuse (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:33.516 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-routes. 2021-02-20T13:06:33.530 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../056-python3-routes_2.4.1-1_all.deb ... 2021-02-20T13:06:33.574 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-routes (2.4.1-1) ... 2021-02-20T13:06:33.695 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mds. 2021-02-20T13:06:33.710 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../051-ceph-mds_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:33.753 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mds (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:33.893 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-02-20T13:06:33.907 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_17.0.0-936-g0d796331-1bionic_all.deb ... 2021-02-20T13:06:33.951 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-dashboard (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:34.265 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package cephadm. 2021-02-20T13:06:34.279 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../052-cephadm_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:34.323 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking cephadm (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:34.654 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jinja2. 2021-02-20T13:06:34.890 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-numpy. 2021-02-20T13:06:34.891 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../058-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2021-02-20T13:06:34.891 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../053-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2021-02-20T13:06:34.892 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-02-20T13:06:34.918 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2021-02-20T13:06:35.032 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-02-20T13:06:35.052 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../054-ceph-mgr-cephadm_17.0.0-936-g0d796331-1bionic_all.deb ... 2021-02-20T13:06:35.090 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-cephadm (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:35.445 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-repoze.lru. 2021-02-20T13:06:35.458 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../055-python3-repoze.lru_0.7-1_all.deb ... 2021-02-20T13:06:35.503 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2021-02-20T13:06:35.546 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-decorator. 2021-02-20T13:06:35.559 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../059-python3-decorator_4.1.2-1_all.deb ... 2021-02-20T13:06:35.604 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-decorator (4.1.2-1) ... 2021-02-20T13:06:35.915 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-scipy. 2021-02-20T13:06:35.930 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../060-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2021-02-20T13:06:35.973 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2021-02-20T13:06:36.147 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-routes. 2021-02-20T13:06:36.163 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../056-python3-routes_2.4.1-1_all.deb ... 2021-02-20T13:06:36.206 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-routes (2.4.1-1) ... 2021-02-20T13:06:36.550 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-02-20T13:06:36.564 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_17.0.0-936-g0d796331-1bionic_all.deb ... 2021-02-20T13:06:36.609 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-dashboard (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:37.519 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-02-20T13:06:37.533 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../061-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2021-02-20T13:06:37.557 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-numpy. 2021-02-20T13:06:37.573 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../058-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2021-02-20T13:06:37.579 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2021-02-20T13:06:37.615 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2021-02-20T13:06:38.050 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-joblib. 2021-02-20T13:06:38.064 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../062-python3-joblib_0.11-1_all.deb ... 2021-02-20T13:06:38.100 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-joblib (0.11-1) ... 2021-02-20T13:06:38.246 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-decorator. 2021-02-20T13:06:38.260 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../059-python3-decorator_4.1.2-1_all.deb ... 2021-02-20T13:06:38.304 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-decorator (4.1.2-1) ... 2021-02-20T13:06:38.446 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-sklearn. 2021-02-20T13:06:38.462 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../063-python3-sklearn_0.19.1-3_all.deb ... 2021-02-20T13:06:38.504 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2021-02-20T13:06:38.599 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-scipy. 2021-02-20T13:06:38.615 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../060-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2021-02-20T13:06:38.657 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2021-02-20T13:06:39.050 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-02-20T13:06:39.068 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../064-ceph-mgr-diskprediction-local_17.0.0-936-g0d796331-1bionic_all.deb ... 2021-02-20T13:06:39.109 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-diskprediction-local (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:40.080 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-02-20T13:06:40.095 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../061-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2021-02-20T13:06:40.172 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2021-02-20T13:06:40.229 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-monotonic. 2021-02-20T13:06:40.244 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../065-python3-monotonic_1.1-2_all.deb ... 2021-02-20T13:06:40.287 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-monotonic (1.1-2) ... 2021-02-20T13:06:40.623 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-fasteners. 2021-02-20T13:06:40.638 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../066-python3-fasteners_0.12.0-3_all.deb ... 2021-02-20T13:06:40.669 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-joblib. 2021-02-20T13:06:40.681 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2021-02-20T13:06:40.683 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../062-python3-joblib_0.11-1_all.deb ... 2021-02-20T13:06:40.736 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-joblib (0.11-1) ... 2021-02-20T13:06:40.967 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-httplib2. 2021-02-20T13:06:40.980 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../067-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2021-02-20T13:06:41.017 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2021-02-20T13:06:41.147 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-sklearn. 2021-02-20T13:06:41.161 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../063-python3-sklearn_0.19.1-3_all.deb ... 2021-02-20T13:06:41.205 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2021-02-20T13:06:41.361 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pyasn1. 2021-02-20T13:06:41.377 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../068-python3-pyasn1_0.4.2-3_all.deb ... 2021-02-20T13:06:41.420 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2021-02-20T13:06:41.740 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-02-20T13:06:41.755 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../069-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2021-02-20T13:06:41.798 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2021-02-20T13:06:41.820 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-02-20T13:06:41.835 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../064-ceph-mgr-diskprediction-local_17.0.0-936-g0d796331-1bionic_all.deb ... 2021-02-20T13:06:41.878 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-diskprediction-local (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:42.117 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-rsa. 2021-02-20T13:06:42.133 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../070-python3-rsa_3.4.2-1_all.deb ... 2021-02-20T13:06:42.176 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-rsa (3.4.2-1) ... 2021-02-20T13:06:42.546 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-sqlalchemy. 2021-02-20T13:06:42.561 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../071-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2021-02-20T13:06:42.604 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-02-20T13:06:43.009 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-monotonic. 2021-02-20T13:06:43.022 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../065-python3-monotonic_1.1-2_all.deb ... 2021-02-20T13:06:43.050 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-oauth2client. 2021-02-20T13:06:43.064 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../072-python3-oauth2client_4.1.2-3_all.deb ... 2021-02-20T13:06:43.065 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-monotonic (1.1-2) ... 2021-02-20T13:06:43.108 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2021-02-20T13:06:43.368 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-fasteners. 2021-02-20T13:06:43.383 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../066-python3-fasteners_0.12.0-3_all.deb ... 2021-02-20T13:06:43.426 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2021-02-20T13:06:43.445 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-setuptools. 2021-02-20T13:06:43.461 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../073-python3-setuptools_39.0.1-2_all.deb ... 2021-02-20T13:06:43.502 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2021-02-20T13:06:43.754 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-httplib2. 2021-02-20T13:06:43.768 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../067-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2021-02-20T13:06:43.813 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2021-02-20T13:06:43.890 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-websocket. 2021-02-20T13:06:43.905 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../074-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2021-02-20T13:06:43.948 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2021-02-20T13:06:44.123 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pyasn1. 2021-02-20T13:06:44.140 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../068-python3-pyasn1_0.4.2-3_all.deb ... 2021-02-20T13:06:44.182 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2021-02-20T13:06:44.267 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-kubernetes. 2021-02-20T13:06:44.287 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../075-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2021-02-20T13:06:44.326 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2021-02-20T13:06:44.527 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-02-20T13:06:44.540 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../069-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2021-02-20T13:06:44.585 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2021-02-20T13:06:44.780 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-02-20T13:06:44.794 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_17.0.0-936-g0d796331-1bionic_all.deb ... 2021-02-20T13:06:44.838 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-k8sevents (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:44.932 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rsa. 2021-02-20T13:06:44.947 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../070-python3-rsa_3.4.2-1_all.deb ... 2021-02-20T13:06:44.990 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rsa (3.4.2-1) ... 2021-02-20T13:06:45.157 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-json-pointer. 2021-02-20T13:06:45.173 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../077-python3-json-pointer_1.10-1_all.deb ... 2021-02-20T13:06:45.215 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-json-pointer (1.10-1) ... 2021-02-20T13:06:45.310 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-sqlalchemy. 2021-02-20T13:06:45.324 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../071-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2021-02-20T13:06:45.377 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-02-20T13:06:45.604 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-jsonpatch. 2021-02-20T13:06:45.618 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../078-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2021-02-20T13:06:45.674 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-02-20T13:06:45.827 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-oauth2client. 2021-02-20T13:06:45.843 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../072-python3-oauth2client_4.1.2-3_all.deb ... 2021-02-20T13:06:45.889 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2021-02-20T13:06:45.982 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-rook. 2021-02-20T13:06:45.998 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../079-ceph-mgr-rook_17.0.0-936-g0d796331-1bionic_all.deb ... 2021-02-20T13:06:46.040 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-rook (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:46.234 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-setuptools. 2021-02-20T13:06:46.353 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libonig4:amd64. 2021-02-20T13:06:46.353 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../073-python3-setuptools_39.0.1-2_all.deb ... 2021-02-20T13:06:46.353 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2021-02-20T13:06:46.368 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../080-libonig4_6.7.0-1_amd64.deb ... 2021-02-20T13:06:46.495 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2021-02-20T13:06:46.696 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-websocket. 2021-02-20T13:06:46.711 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../074-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2021-02-20T13:06:46.754 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2021-02-20T13:06:46.890 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libjq1:amd64. 2021-02-20T13:06:46.907 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../081-libjq1_1.5+dfsg-2_amd64.deb ... 2021-02-20T13:06:46.948 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2021-02-20T13:06:47.077 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-kubernetes. 2021-02-20T13:06:47.091 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../075-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2021-02-20T13:06:47.124 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2021-02-20T13:06:47.192 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package jq. 2021-02-20T13:06:47.206 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../082-jq_1.5+dfsg-2_amd64.deb ... 2021-02-20T13:06:47.251 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking jq (1.5+dfsg-2) ... 2021-02-20T13:06:47.503 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package socat. 2021-02-20T13:06:47.518 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../083-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2021-02-20T13:06:47.520 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-02-20T13:06:47.535 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_17.0.0-936-g0d796331-1bionic_all.deb ... 2021-02-20T13:06:47.561 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2021-02-20T13:06:47.578 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-k8sevents (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:47.956 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-json-pointer. 2021-02-20T13:06:47.977 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../077-python3-json-pointer_1.10-1_all.deb ... 2021-02-20T13:06:47.990 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-02-20T13:06:48.004 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../084-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2021-02-20T13:06:48.017 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-json-pointer (1.10-1) ... 2021-02-20T13:06:48.048 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-02-20T13:06:48.355 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jsonpatch. 2021-02-20T13:06:48.356 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package xmlstarlet. 2021-02-20T13:06:48.366 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2_amd64.deb ... 2021-02-20T13:06:48.370 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../078-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2021-02-20T13:06:48.410 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2021-02-20T13:06:48.416 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-02-20T13:06:48.741 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-rook. 2021-02-20T13:06:48.756 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../079-ceph-mgr-rook_17.0.0-936-g0d796331-1bionic_all.deb ... 2021-02-20T13:06:48.787 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-test. 2021-02-20T13:06:48.795 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../086-ceph-test_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:48.807 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-rook (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:48.838 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-test (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:49.152 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libonig4:amd64. 2021-02-20T13:06:49.172 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../080-libonig4_6.7.0-1_amd64.deb ... 2021-02-20T13:06:49.301 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2021-02-20T13:06:49.665 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libjq1:amd64. 2021-02-20T13:06:49.680 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../081-libjq1_1.5+dfsg-2_amd64.deb ... 2021-02-20T13:06:49.724 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2021-02-20T13:06:50.010 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package jq. 2021-02-20T13:06:50.024 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../082-jq_1.5+dfsg-2_amd64.deb ... 2021-02-20T13:06:50.068 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking jq (1.5+dfsg-2) ... 2021-02-20T13:06:50.320 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package socat. 2021-02-20T13:06:50.337 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../083-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2021-02-20T13:06:50.379 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2021-02-20T13:06:50.782 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-02-20T13:06:50.796 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../084-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2021-02-20T13:06:50.841 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-02-20T13:06:51.135 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package xmlstarlet. 2021-02-20T13:06:51.151 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2_amd64.deb ... 2021-02-20T13:06:51.202 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2021-02-20T13:06:51.573 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-test. 2021-02-20T13:06:51.587 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../086-ceph-test_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:51.631 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-test (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:52.184 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libcephfs-dev. 2021-02-20T13:06:52.198 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../087-libcephfs-dev_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:52.259 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libcephfs-dev (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:52.554 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package nvme-cli. 2021-02-20T13:06:52.571 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../088-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2021-02-20T13:06:52.620 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2021-02-20T13:06:52.966 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python-attr. 2021-02-20T13:06:52.979 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../089-python-attr_17.4.0-2_all.deb ... 2021-02-20T13:06:53.024 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python-attr (17.4.0-2) ... 2021-02-20T13:06:53.327 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python-funcsigs. 2021-02-20T13:06:53.344 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../090-python-funcsigs_1.0.2-4_all.deb ... 2021-02-20T13:06:53.386 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2021-02-20T13:06:53.689 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python-pluggy. 2021-02-20T13:06:53.706 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../091-python-pluggy_0.6.0-1_all.deb ... 2021-02-20T13:06:53.747 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python-pluggy (0.6.0-1) ... 2021-02-20T13:06:54.066 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python-py. 2021-02-20T13:06:54.081 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../092-python-py_1.5.2-1_all.deb ... 2021-02-20T13:06:54.124 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python-py (1.5.2-1) ... 2021-02-20T13:06:54.478 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python-pytest. 2021-02-20T13:06:54.493 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../093-python-pytest_3.3.2-2_all.deb ... 2021-02-20T13:06:54.536 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python-pytest (3.3.2-2) ... 2021-02-20T13:06:54.881 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-attr. 2021-02-20T13:06:54.898 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../094-python3-attr_17.4.0-2_all.deb ... 2021-02-20T13:06:54.939 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-attr (17.4.0-2) ... 2021-02-20T13:06:55.069 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libcephfs-dev. 2021-02-20T13:06:55.084 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../087-libcephfs-dev_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:06:55.128 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libcephfs-dev (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:06:55.242 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cycler. 2021-02-20T13:06:55.259 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../095-python3-cycler_0.10.0-1_all.deb ... 2021-02-20T13:06:55.301 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cycler (0.10.0-1) ... 2021-02-20T13:06:55.422 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package nvme-cli. 2021-02-20T13:06:55.438 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../088-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2021-02-20T13:06:55.489 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2021-02-20T13:06:55.620 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-webencodings. 2021-02-20T13:06:55.636 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../096-python3-webencodings_0.5-2_all.deb ... 2021-02-20T13:06:55.678 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-webencodings (0.5-2) ... 2021-02-20T13:06:55.835 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python-attr. 2021-02-20T13:06:55.850 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../089-python-attr_17.4.0-2_all.deb ... 2021-02-20T13:06:55.893 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python-attr (17.4.0-2) ... 2021-02-20T13:06:56.023 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-html5lib. 2021-02-20T13:06:56.038 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../097-python3-html5lib_0.999999999-1_all.deb ... 2021-02-20T13:06:56.081 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2021-02-20T13:06:56.339 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python-funcsigs. 2021-02-20T13:06:56.354 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../090-python-funcsigs_1.0.2-4_all.deb ... 2021-02-20T13:06:56.397 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2021-02-20T13:06:56.401 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-02-20T13:06:56.418 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../098-python3-lxml_4.2.1-1ubuntu0.3_amd64.deb ... 2021-02-20T13:06:56.466 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-02-20T13:06:56.700 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python-pluggy. 2021-02-20T13:06:56.713 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../091-python-pluggy_0.6.0-1_all.deb ... 2021-02-20T13:06:56.758 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python-pluggy (0.6.0-1) ... 2021-02-20T13:06:56.930 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pyparsing. 2021-02-20T13:06:56.946 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../099-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2021-02-20T13:06:56.988 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2021-02-20T13:06:57.069 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python-py. 2021-02-20T13:06:57.083 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../092-python-py_1.5.2-1_all.deb ... 2021-02-20T13:06:57.127 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python-py (1.5.2-1) ... 2021-02-20T13:06:57.308 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-tz. 2021-02-20T13:06:57.322 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../100-python3-tz_2018.3-2_all.deb ... 2021-02-20T13:06:57.366 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-tz (2018.3-2) ... 2021-02-20T13:06:57.497 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python-pytest. 2021-02-20T13:06:57.514 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../093-python-pytest_3.3.2-2_all.deb ... 2021-02-20T13:06:57.579 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python-pytest (3.3.2-2) ... 2021-02-20T13:06:57.660 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-matplotlib. 2021-02-20T13:06:57.675 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../101-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2021-02-20T13:06:57.743 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2021-02-20T13:06:57.893 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-attr. 2021-02-20T13:06:57.909 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../094-python3-attr_17.4.0-2_all.deb ... 2021-02-20T13:06:57.951 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-attr (17.4.0-2) ... 2021-02-20T13:06:58.296 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cycler. 2021-02-20T13:06:58.311 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../095-python3-cycler_0.10.0-1_all.deb ... 2021-02-20T13:06:58.354 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cycler (0.10.0-1) ... 2021-02-20T13:06:58.382 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-olefile. 2021-02-20T13:06:58.397 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../102-python3-olefile_0.45.1-1_all.deb ... 2021-02-20T13:06:58.450 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-olefile (0.45.1-1) ... 2021-02-20T13:06:58.657 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-webencodings. 2021-02-20T13:06:58.671 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../096-python3-webencodings_0.5-2_all.deb ... 2021-02-20T13:06:58.715 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-webencodings (0.5-2) ... 2021-02-20T13:06:58.752 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pastescript. 2021-02-20T13:06:58.768 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-2_all.deb ... 2021-02-20T13:06:58.861 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2021-02-20T13:06:59.019 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-html5lib. 2021-02-20T13:06:59.032 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../097-python3-html5lib_0.999999999-1_all.deb ... 2021-02-20T13:06:59.077 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2021-02-20T13:06:59.214 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pil:amd64. 2021-02-20T13:06:59.230 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../104-python3-pil_5.1.0-1ubuntu0.4_amd64.deb ... 2021-02-20T13:06:59.272 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.4) ... 2021-02-20T13:06:59.472 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-02-20T13:06:59.487 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../098-python3-lxml_4.2.1-1ubuntu0.3_amd64.deb ... 2021-02-20T13:06:59.531 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-02-20T13:06:59.626 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pluggy. 2021-02-20T13:06:59.643 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../105-python3-pluggy_0.6.0-1_all.deb ... 2021-02-20T13:06:59.684 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2021-02-20T13:07:00.011 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-py. 2021-02-20T13:07:00.026 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../106-python3-py_1.5.2-1_all.deb ... 2021-02-20T13:07:00.051 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pyparsing. 2021-02-20T13:07:00.066 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../099-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2021-02-20T13:07:00.069 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-py (1.5.2-1) ... 2021-02-20T13:07:00.116 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2021-02-20T13:07:00.399 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pyinotify. 2021-02-20T13:07:00.415 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1_all.deb ... 2021-02-20T13:07:00.421 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-tz. 2021-02-20T13:07:00.438 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../100-python3-tz_2018.3-2_all.deb ... 2021-02-20T13:07:00.457 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2021-02-20T13:07:00.479 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-tz (2018.3-2) ... 2021-02-20T13:07:00.759 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pytest. 2021-02-20T13:07:00.776 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../108-python3-pytest_3.3.2-2_all.deb ... 2021-02-20T13:07:00.782 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-matplotlib. 2021-02-20T13:07:00.796 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../101-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2021-02-20T13:07:00.823 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pytest (3.3.2-2) ... 2021-02-20T13:07:00.840 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2021-02-20T13:07:01.171 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-simplejson. 2021-02-20T13:07:01.187 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../109-python3-simplejson_3.13.2-1_amd64.deb ... 2021-02-20T13:07:01.229 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2021-02-20T13:07:01.523 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-olefile. 2021-02-20T13:07:01.536 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../102-python3-olefile_0.45.1-1_all.deb ... 2021-02-20T13:07:01.574 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-tk:amd64. 2021-02-20T13:07:01.580 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-olefile (0.45.1-1) ... 2021-02-20T13:07:01.590 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../110-python3-tk_3.6.9-1~18.04_amd64.deb ... 2021-02-20T13:07:01.640 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2021-02-20T13:07:01.899 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pastescript. 2021-02-20T13:07:01.916 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-2_all.deb ... 2021-02-20T13:07:01.991 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2021-02-20T13:07:02.235 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package radosgw. 2021-02-20T13:07:02.250 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../111-radosgw_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:07:02.293 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking radosgw (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:02.336 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pil:amd64. 2021-02-20T13:07:02.352 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../104-python3-pil_5.1.0-1ubuntu0.4_amd64.deb ... 2021-02-20T13:07:02.395 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.4) ... 2021-02-20T13:07:02.748 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pluggy. 2021-02-20T13:07:02.764 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../105-python3-pluggy_0.6.0-1_all.deb ... 2021-02-20T13:07:02.814 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2021-02-20T13:07:03.159 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-py. 2021-02-20T13:07:03.174 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../106-python3-py_1.5.2-1_all.deb ... 2021-02-20T13:07:03.217 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-py (1.5.2-1) ... 2021-02-20T13:07:03.355 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package rbd-fuse. 2021-02-20T13:07:03.372 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../112-rbd-fuse_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:07:03.413 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking rbd-fuse (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:03.537 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pyinotify. 2021-02-20T13:07:03.551 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1_all.deb ... 2021-02-20T13:07:03.595 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2021-02-20T13:07:03.758 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package rbd-mirror. 2021-02-20T13:07:03.772 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../113-rbd-mirror_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:07:03.824 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking rbd-mirror (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:03.932 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pytest. 2021-02-20T13:07:03.947 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../108-python3-pytest_3.3.2-2_all.deb ... 2021-02-20T13:07:03.990 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pytest (3.3.2-2) ... 2021-02-20T13:07:04.327 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-simplejson. 2021-02-20T13:07:04.344 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../109-python3-simplejson_3.13.2-1_amd64.deb ... 2021-02-20T13:07:04.385 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2021-02-20T13:07:04.437 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-logutils (0.3.3-5) ... 2021-02-20T13:07:04.730 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-tk:amd64. 2021-02-20T13:07:04.738 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-02-20T13:07:04.746 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../110-python3-tk_3.6.9-1~18.04_amd64.deb ... 2021-02-20T13:07:04.788 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2021-02-20T13:07:05.098 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2021-02-20T13:07:05.224 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-02-20T13:07:05.267 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package radosgw. 2021-02-20T13:07:05.284 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../111-radosgw_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:07:05.325 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking radosgw (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:05.843 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.4) ... 2021-02-20T13:07:06.236 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2021-02-20T13:07:06.370 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package rbd-fuse. 2021-02-20T13:07:06.397 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../112-rbd-fuse_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:07:06.529 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-webencodings (0.5-2) ... 2021-02-20T13:07:06.803 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking rbd-fuse (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:06.847 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python-attr (17.4.0-2) ... 2021-02-20T13:07:07.056 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2021-02-20T13:07:07.140 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package rbd-mirror. 2021-02-20T13:07:07.155 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../113-rbd-mirror_17.0.0-936-g0d796331-1bionic_amd64.deb ... 2021-02-20T13:07:07.182 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-py (1.5.2-1) ... 2021-02-20T13:07:07.198 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking rbd-mirror (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:07.534 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-waitress (1.0.1-1) ... 2021-02-20T13:07:07.835 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-logutils (0.3.3-5) ... 2021-02-20T13:07:07.862 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2021-02-20T13:07:08.145 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-02-20T13:07:08.170 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-olefile (0.45.1-1) ... 2021-02-20T13:07:08.482 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-ceph-argparse (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:08.514 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2021-02-20T13:07:08.640 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-02-20T13:07:08.791 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-tempita (0.5.2-2) ... 2021-02-20T13:07:09.117 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2021-02-20T13:07:09.259 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2021-02-20T13:07:09.267 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.4) ... 2021-02-20T13:07:09.669 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2021-02-20T13:07:09.970 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-webencodings (0.5-2) ... 2021-02-20T13:07:10.179 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2021-02-20T13:07:10.280 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python-attr (17.4.0-2) ... 2021-02-20T13:07:10.488 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2021-02-20T13:07:10.498 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2021-02-20T13:07:10.666 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-py (1.5.2-1) ... 2021-02-20T13:07:10.773 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python-py (1.5.2-1) ... 2021-02-20T13:07:11.017 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-simplejson (3.13.2-1) ... 2021-02-20T13:07:11.017 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-waitress (1.0.1-1) ... 2021-02-20T13:07:11.352 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2021-02-20T13:07:11.376 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-json-pointer (1.10-1) ... 2021-02-20T13:07:11.467 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2021-02-20T13:07:11.653 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-olefile (0.45.1-1) ... 2021-02-20T13:07:11.694 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2021-02-20T13:07:11.820 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python-pluggy (0.6.0-1) ... 2021-02-20T13:07:11.938 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-ceph-argparse (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:12.256 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-tempita (0.5.2-2) ... 2021-02-20T13:07:12.413 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2021-02-20T13:07:12.550 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2021-02-20T13:07:12.675 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2021-02-20T13:07:12.715 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2021-02-20T13:07:13.024 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cycler (0.10.0-1) ... 2021-02-20T13:07:13.316 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2021-02-20T13:07:13.603 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2021-02-20T13:07:13.693 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-bs4 (4.6.0-1) ... 2021-02-20T13:07:13.921 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2021-02-20T13:07:14.019 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pluggy (0.6.0-1) ... 2021-02-20T13:07:14.224 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python-py (1.5.2-1) ... 2021-02-20T13:07:14.312 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up cephadm (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:14.483 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-simplejson (3.13.2-1) ... 2021-02-20T13:07:14.834 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-json-pointer (1.10-1) ... 2021-02-20T13:07:14.924 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2021-02-20T13:07:14.947 INFO:teuthology.orchestra.run.smithi183.stdout:Adding system user cephadm....done 2021-02-20T13:07:15.135 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2021-02-20T13:07:15.169 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2021-02-20T13:07:15.321 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python-pluggy (0.6.0-1) ... 2021-02-20T13:07:15.419 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-02-20T13:07:15.522 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2021-02-20T13:07:15.545 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2021-02-20T13:07:15.679 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2021-02-20T13:07:15.815 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2021-02-20T13:07:16.048 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-prettytable (0.7.2-3) ... 2021-02-20T13:07:16.141 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cycler (0.10.0-1) ... 2021-02-20T13:07:16.348 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2021-02-20T13:07:16.467 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2021-02-20T13:07:16.476 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2021-02-20T13:07:16.835 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-bs4 (4.6.0-1) ... 2021-02-20T13:07:16.851 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-ceph-common (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:17.178 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-monotonic (1.1-2) ... 2021-02-20T13:07:17.186 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pluggy (0.6.0-1) ... 2021-02-20T13:07:17.461 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2021-02-20T13:07:17.496 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up cephadm (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:17.588 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-02-20T13:07:17.950 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-html5lib (0.999999999-1) ... 2021-02-20T13:07:18.172 INFO:teuthology.orchestra.run.smithi170.stdout:Adding system user cephadm....done 2021-02-20T13:07:18.317 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-02-20T13:07:18.353 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2021-02-20T13:07:18.424 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2021-02-20T13:07:18.458 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2021-02-20T13:07:18.646 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-02-20T13:07:18.694 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-repoze.lru (0.7-1) ... 2021-02-20T13:07:18.780 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2021-02-20T13:07:18.915 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2021-02-20T13:07:18.995 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2021-02-20T13:07:19.266 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-prettytable (0.7.2-3) ... 2021-02-20T13:07:19.329 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2021-02-20T13:07:19.567 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2021-02-20T13:07:19.680 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-attr (17.4.0-2) ... 2021-02-20T13:07:19.693 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2021-02-20T13:07:20.007 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python-funcsigs (1.0.2-4) ... 2021-02-20T13:07:20.037 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-ceph-common (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:20.208 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libjaeger (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:20.359 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-02-20T13:07:20.380 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-monotonic (1.1-2) ... 2021-02-20T13:07:20.689 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2021-02-20T13:07:20.778 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2021-02-20T13:07:20.815 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-02-20T13:07:20.868 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-02-20T13:07:21.165 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up liboath0 (2.6.1-1) ... 2021-02-20T13:07:21.177 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-html5lib (0.999999999-1) ... 2021-02-20T13:07:21.326 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-02-20T13:07:21.449 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2021-02-20T13:07:21.528 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-02-20T13:07:21.618 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2021-02-20T13:07:21.650 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2021-02-20T13:07:21.701 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2021-02-20T13:07:21.829 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-setuptools (39.0.1-2) ... 2021-02-20T13:07:21.898 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-repoze.lru (0.7-1) ... 2021-02-20T13:07:22.207 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2021-02-20T13:07:22.221 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2021-02-20T13:07:22.380 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-decorator (4.1.2-1) ... 2021-02-20T13:07:22.517 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2021-02-20T13:07:22.673 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-tz (2018.3-2) ... 2021-02-20T13:07:22.868 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-attr (17.4.0-2) ... 2021-02-20T13:07:22.974 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2021-02-20T13:07:23.161 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python-funcsigs (1.0.2-4) ... 2021-02-20T13:07:23.267 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2021-02-20T13:07:23.355 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libjaeger (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:23.489 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-02-20T13:07:23.559 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-joblib (0.11-1) ... 2021-02-20T13:07:23.899 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2021-02-20T13:07:23.944 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-routes (2.4.1-1) ... 2021-02-20T13:07:23.990 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-02-20T13:07:24.227 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up liboath0 (2.6.1-1) ... 2021-02-20T13:07:24.245 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2021-02-20T13:07:24.395 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-02-20T13:07:24.506 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2021-02-20T13:07:24.563 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-dateutil (2.6.1-1) ... 2021-02-20T13:07:24.740 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2021-02-20T13:07:24.841 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-setuptools (39.0.1-2) ... 2021-02-20T13:07:24.898 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-02-20T13:07:25.213 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2021-02-20T13:07:25.267 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-fasteners (0.12.0-3) ... 2021-02-20T13:07:25.335 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-decorator (4.1.2-1) ... 2021-02-20T13:07:25.585 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up jq (1.5+dfsg-2) ... 2021-02-20T13:07:25.645 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-tz (2018.3-2) ... 2021-02-20T13:07:25.736 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2021-02-20T13:07:25.937 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2021-02-20T13:07:26.214 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2021-02-20T13:07:26.498 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-joblib (0.11-1) ... 2021-02-20T13:07:26.875 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-routes (2.4.1-1) ... 2021-02-20T13:07:26.899 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-02-20T13:07:27.176 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2021-02-20T13:07:27.234 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2021-02-20T13:07:27.477 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-dateutil (2.6.1-1) ... 2021-02-20T13:07:27.577 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2021-02-20T13:07:27.703 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pytest (3.3.2-2) ... 2021-02-20T13:07:27.813 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-02-20T13:07:28.054 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python-pytest (3.3.2-2) ... 2021-02-20T13:07:28.148 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-fasteners (0.12.0-3) ... 2021-02-20T13:07:28.314 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2021-02-20T13:07:28.433 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up jq (1.5+dfsg-2) ... 2021-02-20T13:07:28.440 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up xmlstarlet (1.6.1-2) ... 2021-02-20T13:07:28.559 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2021-02-20T13:07:28.565 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librados2 (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:28.692 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-rsa (3.4.2-1) ... 2021-02-20T13:07:29.018 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libcephfs2 (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:29.152 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libcephfs-dev (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:29.278 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-rados (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:29.404 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2021-02-20T13:07:29.730 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2021-02-20T13:07:29.750 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-02-20T13:07:30.023 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2021-02-20T13:07:30.076 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2021-02-20T13:07:30.402 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2021-02-20T13:07:30.468 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2021-02-20T13:07:30.528 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pytest (3.3.2-2) ... 2021-02-20T13:07:30.904 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python-pytest (3.3.2-2) ... 2021-02-20T13:07:31.139 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2021-02-20T13:07:31.241 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up xmlstarlet (1.6.1-2) ... 2021-02-20T13:07:31.312 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-sklearn (0.19.1-3) ... 2021-02-20T13:07:31.366 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librados2 (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:31.476 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rsa (3.4.2-1) ... 2021-02-20T13:07:31.785 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libcephfs2 (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:31.911 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libcephfs-dev (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:32.045 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rados (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:32.152 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2021-02-20T13:07:32.163 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2021-02-20T13:07:32.462 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libradosstriper1 (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:32.472 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2021-02-20T13:07:32.588 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cephfs (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:32.722 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librgw2 (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:32.749 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2021-02-20T13:07:32.848 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-fuse (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:33.068 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-02-20T13:07:33.069 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-02-20T13:07:33.160 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2021-02-20T13:07:33.358 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librbd1 (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:33.526 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pecan (1.2.1-2) ... 2021-02-20T13:07:33.632 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2021-02-20T13:07:33.665 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2021-02-20T13:07:33.970 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2021-02-20T13:07:34.030 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-sklearn (0.19.1-3) ... 2021-02-20T13:07:34.648 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-modules-core (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:34.774 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pastescript (2.0.2-2) ... 2021-02-20T13:07:34.845 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2021-02-20T13:07:35.092 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-rgw (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:35.138 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libradosstriper1 (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:35.234 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up rbd-fuse (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:35.239 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cephfs (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:35.341 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librgw2 (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:35.369 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-rbd (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:35.450 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-fuse (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:35.495 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-common (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:35.647 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-02-20T13:07:35.647 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-02-20T13:07:35.926 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librbd1 (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:36.046 INFO:teuthology.orchestra.run.smithi183.stdout:Adding group ceph....done 2021-02-20T13:07:36.069 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pecan (1.2.1-2) ... 2021-02-20T13:07:36.158 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2021-02-20T13:07:36.191 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2021-02-20T13:07:36.423 INFO:teuthology.orchestra.run.smithi183.stdout:Adding system user ceph....done 2021-02-20T13:07:36.505 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2021-02-20T13:07:36.582 INFO:teuthology.orchestra.run.smithi183.stdout:Setting system user ceph properties....done 2021-02-20T13:07:36.741 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-02-20T13:07:36.960 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-02-20T13:07:37.246 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up radosgw (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:37.476 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-02-20T13:07:37.476 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-02-20T13:07:37.566 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-modules-core (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:37.667 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pastescript (2.0.2-2) ... 2021-02-20T13:07:37.866 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up rbd-mirror (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:37.994 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rgw (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:38.080 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2021-02-20T13:07:38.081 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2021-02-20T13:07:38.119 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up rbd-fuse (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:38.245 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rbd (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:38.351 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-test (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:38.363 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-common (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:38.477 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-base (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:38.802 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-02-20T13:07:38.839 INFO:teuthology.orchestra.run.smithi170.stdout:Adding group ceph....done 2021-02-20T13:07:39.070 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:39.216 INFO:teuthology.orchestra.run.smithi170.stdout:Adding system user ceph....done 2021-02-20T13:07:39.267 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-02-20T13:07:39.267 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-02-20T13:07:39.409 INFO:teuthology.orchestra.run.smithi170.stdout:Setting system user ceph properties....done 2021-02-20T13:07:39.521 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-diskprediction-local (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:39.569 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-02-20T13:07:39.639 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-osd (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:39.772 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-02-20T13:07:40.042 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-02-20T13:07:40.043 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-02-20T13:07:40.065 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up radosgw (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:40.309 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-02-20T13:07:40.309 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-02-20T13:07:40.325 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-cephadm (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:40.459 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-rook (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:40.601 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mds (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:40.743 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up rbd-mirror (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:40.828 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-02-20T13:07:40.829 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-02-20T13:07:40.966 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2021-02-20T13:07:40.966 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2021-02-20T13:07:41.119 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mon (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:41.236 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-test (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:41.329 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-02-20T13:07:41.330 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-02-20T13:07:41.362 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-base (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:41.580 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-k8sevents (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:41.688 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-dashboard (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:41.690 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-02-20T13:07:41.806 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:41.907 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for systemd (237-3ubuntu10.44) ... 2021-02-20T13:07:41.988 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:42.085 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-02-20T13:07:42.191 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-02-20T13:07:42.191 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-02-20T13:07:42.482 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-diskprediction-local (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:42.625 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-osd (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:43.077 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-02-20T13:07:43.078 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-02-20T13:07:43.394 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-cephadm (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:43.562 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-rook (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:43.706 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mds (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:43.914 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-02-20T13:07:43.915 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-02-20T13:07:44.186 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mon (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:44.274 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2021-02-20T13:07:44.306 INFO:teuthology.orchestra.run.smithi183.stdout:ureadahead will be reprofiled on next reboot 2021-02-20T13:07:44.354 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.4) ... 2021-02-20T13:07:44.374 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-02-20T13:07:44.375 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-02-20T13:07:44.682 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-k8sevents (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:44.783 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-dashboard (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:44.848 INFO:teuthology.orchestra.run.smithi183.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-02-20T13:07:44.851 DEBUG:teuthology.parallel:result is None 2021-02-20T13:07:44.901 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph (17.0.0-936-g0d796331-1bionic) ... 2021-02-20T13:07:45.019 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for systemd (237-3ubuntu10.44) ... 2021-02-20T13:07:45.245 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-02-20T13:07:47.323 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2021-02-20T13:07:47.458 INFO:teuthology.orchestra.run.smithi170.stdout:ureadahead will be reprofiled on next reboot 2021-02-20T13:07:47.509 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.4) ... 2021-02-20T13:07:47.935 INFO:teuthology.orchestra.run.smithi170.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-02-20T13:07:47.938 DEBUG:teuthology.parallel:result is None 2021-02-20T13:07:47.939 DEBUG:teuthology.orchestra.run.smithi170:> sudo lsb_release -is 2021-02-20T13:07:47.989 INFO:teuthology.orchestra.run.smithi170.stdout:Ubuntu 2021-02-20T13:07:47.994 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-02-20T13:07:47.995 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=0d796331675b92762d821b40e70f6598bcea1178 2021-02-20T13:07:48.162 DEBUG:teuthology.orchestra.run.smithi170:> dpkg-query -W -f '${Version}' ceph 2021-02-20T13:07:48.186 INFO:teuthology.orchestra.run.smithi170.stdout:17.0.0-936-g0d796331-1bionic 2021-02-20T13:07:48.187 INFO:teuthology.packaging:The installed version of ceph is 17.0.0-936-g0d796331-1bionic 2021-02-20T13:07:48.188 INFO:teuthology.task.install:The correct ceph version 17.0.0-936-g0d796331-1bionic is installed. 2021-02-20T13:07:48.188 DEBUG:teuthology.orchestra.run.smithi183:> sudo lsb_release -is 2021-02-20T13:07:48.248 INFO:teuthology.orchestra.run.smithi183.stdout:Ubuntu 2021-02-20T13:07:48.253 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-02-20T13:07:48.254 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=0d796331675b92762d821b40e70f6598bcea1178 2021-02-20T13:07:48.427 DEBUG:teuthology.orchestra.run.smithi183:> dpkg-query -W -f '${Version}' ceph 2021-02-20T13:07:48.453 INFO:teuthology.orchestra.run.smithi183.stdout:17.0.0-936-g0d796331-1bionic 2021-02-20T13:07:48.453 INFO:teuthology.packaging:The installed version of ceph is 17.0.0-936-g0d796331-1bionic 2021-02-20T13:07:48.454 INFO:teuthology.task.install:The correct ceph version 17.0.0-936-g0d796331-1bionic is installed. 2021-02-20T13:07:48.454 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2021-02-20T13:07:48.459 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:07:48.460 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-02-20T13:07:48.475 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:07:48.476 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-02-20T13:07:48.509 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2021-02-20T13:07:48.511 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:07:48.511 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/daemon-helper 2021-02-20T13:07:48.530 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-02-20T13:07:48.585 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:07:48.585 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2021-02-20T13:07:48.602 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-02-20T13:07:48.656 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2021-02-20T13:07:48.657 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:07:48.657 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/adjust-ulimits 2021-02-20T13:07:48.718 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-02-20T13:07:48.774 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:07:48.775 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2021-02-20T13:07:48.790 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-02-20T13:07:48.849 INFO:teuthology.run_tasks:Running task ceph... 2021-02-20T13:07:49.005 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2021-02-20T13:07:49.006 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 777 /var/log/ceph 2021-02-20T13:07:49.010 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /var/log/ceph 2021-02-20T13:07:49.029 INFO:tasks.ceph:Disabling ceph logrotate... 2021-02-20T13:07:49.029 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/logrotate.d/ceph 2021-02-20T13:07:49.074 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/logrotate.d/ceph 2021-02-20T13:07:49.089 INFO:tasks.ceph:Creating extra log directories... 2021-02-20T13:07:49.090 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-02-20T13:07:49.130 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-02-20T13:07:49.220 INFO:tasks.ceph:Creating ceph cluster cluster1... 2021-02-20T13:07:49.220 INFO:tasks.ceph:config {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'none'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(OSD_SLOW_PING_TIME'], 'cpu_profile': set(), 'cluster': 'cluster1', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2021-02-20T13:07:49.220 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/trociny-2021-02-20_12:31:02-rbd-wip-mgolub-testing-distro-basic-smithi/5898608', 'branch': 'wip-mgolub-testing', 'description': 'rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}', 'email': 'to.my.trociny@gmail.com', 'first_in_suite': False, 'job_id': '5898608', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': 'run two ceph clusters and install rbd-mirror'}, {'desc': 'configure the permissions for client.mirror'}, {'desc': '2 ceph clusters with 1 mon and 3 osds each'}, {'desc': 'run four rbd-mirror daemons per cluster'}, {'desc': 'run the rbd_mirror_stress.sh workunit to test the rbd-mirror daemon'}], 'name': 'trociny-2021-02-20_12:31:02-rbd-wip-mgolub-testing-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '18.04', 'overrides': {'admin_socket': {'branch': 'wip-mgolub-testing'}, 'ceph': {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'none'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(OSD_SLOW_PING_TIME'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '0d796331675b92762d821b40e70f6598bcea1178'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': '0d796331675b92762d821b40e70f6598bcea1178'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mgolub-testing', 'sha1': '0d796331675b92762d821b40e70f6598bcea1178'}}, 'owner': 'scheduled_trociny@teuthology', 'priority': 50, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'], ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']], 'sha1': '0d796331675b92762d821b40e70f6598bcea1178', 'sleep_before_teardown': 0, 'suite': 'rbd', 'suite_branch': 'wip-mgolub-testing', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_0d796331675b92762d821b40e70f6598bcea1178/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '0d796331675b92762d821b40e70f6598bcea1178', 'targets': {'smithi170.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDfrY9y6okoGGmSdGP0lWbQ4ocf5wj8XLxLWzIzKeOEY6RLGsDGfOqyKmBHiiztSFrBhM9jjTJxOvvPlTSrU9d2Dgm1QmuEjYA1JLPSaO8piMG/OlaSjI4PHxt9nxvTu77gXoXfXpK6Qo+ZUqZRv3OF9nApXNfTuDTYec8solgHl1hrUrW+n+Ow9gmRFNrBkDqVvEnYG7Xg+EQEm5aelmrdIhIdLh5Mjum14tIxk4i/1a2umeJOt88yzRzjLS2+H2/EXUn5QOGGpkz6mCS1Xd5L9UP4JVeYG9wdykGBhPJykA6OCzH5JEbGUEvqoSruNpN65AWm8PTlQ9hYB4OoK0VX', 'smithi183.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC7bsnX0wXavEUXQ1xu3LEdpYndhq6jmv7ggYLSbXPw77TjZBIbrBrtxwh0XeWqiP9d2H0n0hf/drGz5LjqiplvzH9leTtkoAE3Y+0niWLqunqLFKlAqLRKPSrZKtJtc0kDrn34lpO5aORxGAD/ZfqsRrXLLNYgD3VV9XLNPPldmEFTjPOXJa1R2mAhDDfEFXuj/pZxhL+LlJytgvOI6EGcrYgixGtDVoQ2b8zbsz/4Z/0YN/TvhEE+TZs22LKfPiivEoAMv21FVua6WDv9WX526ChOhnkLnBFJXRavZG4r14sd4+2aWAgd+cS47CDYTuJQxs2hdzY8crT3ZRJPsIRL'}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': ['rbd-mirror'], 'sha1': '0d796331675b92762d821b40e70f6598bcea1178'}}, {'ceph': {'cluster': 'cluster1', 'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'none'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(OSD_SLOW_PING_TIME'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '0d796331675b92762d821b40e70f6598bcea1178'}}, {'ceph': {'cluster': 'cluster2'}}, {'exec': {'cluster1.client.mirror.0': ["sudo ceph --cluster cluster1 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'"]}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.3', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.3', 'thrash': True}}, {'rbd-mirror-thrash': {'cluster': 'cluster1'}}, {'rbd-mirror-thrash': {'cluster': 'cluster2'}}, {'workunit': {'clients': {'cluster1.client.mirror': ['rbd/rbd_mirror_stress.sh']}, 'env': {'CEPH_ARGS': '', 'MIRROR_IMAGE_MODE': 'snapshot', 'MIRROR_POOL_MODE': 'image', 'RBD_IMAGE_FEATURES': 'layering,exclusive-lock', 'RBD_MIRROR_INSTANCES': '4', 'RBD_MIRROR_USE_EXISTING_CLUSTER': '1', 'RBD_MIRROR_USE_RBD_MIRROR': '1'}, 'timeout': '6h'}}], 'teuthology_branch': 'master', 'tube': 'smithi', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.29353'} 2021-02-20T13:07:49.222 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/cluster1.data 2021-02-20T13:07:49.237 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/cluster1.data 2021-02-20T13:07:49.254 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m0777 -- /var/run/ceph 2021-02-20T13:07:49.309 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m0777 -- /var/run/ceph 2021-02-20T13:07:49.325 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:07:49.326 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2021-02-20T13:07:49.387 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-02-20T13:07:49.388 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-02-20T13:07:49.432 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-02-20T13:07:49.433 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T13:07:49.433 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 341 Links: 1 2021-02-20T13:07:49.434 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T13:07:49.434 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2021-02-20 13:04:58.256205647 +0000 2021-02-20T13:07:49.435 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2021-02-20 13:04:58.136205624 +0000 2021-02-20T13:07:49.435 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2021-02-20 13:04:58.136205624 +0000 2021-02-20T13:07:49.435 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2021-02-20T13:07:49.442 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2021-02-20T13:07:49.443 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2021-02-20T13:07:49.443 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000222336 s, 2.3 MB/s 2021-02-20T13:07:49.446 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-02-20T13:07:49.495 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-02-20T13:07:49.496 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T13:07:49.496 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 347 Links: 1 2021-02-20T13:07:49.497 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T13:07:49.497 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2021-02-20 13:04:58.476205688 +0000 2021-02-20T13:07:49.497 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2021-02-20 13:04:58.384205670 +0000 2021-02-20T13:07:49.498 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2021-02-20 13:04:58.384205670 +0000 2021-02-20T13:07:49.498 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2021-02-20T13:07:49.503 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2021-02-20T13:07:49.504 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2021-02-20T13:07:49.504 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000238907 s, 2.1 MB/s 2021-02-20T13:07:49.508 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-02-20T13:07:49.559 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-02-20T13:07:49.559 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T13:07:49.559 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 353 Links: 1 2021-02-20T13:07:49.560 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T13:07:49.560 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2021-02-20 13:04:58.712205732 +0000 2021-02-20T13:07:49.562 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2021-02-20 13:04:58.584205708 +0000 2021-02-20T13:07:49.563 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2021-02-20 13:04:58.584205708 +0000 2021-02-20T13:07:49.569 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2021-02-20T13:07:49.572 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2021-02-20T13:07:49.572 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2021-02-20T13:07:49.573 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.00027212 s, 1.9 MB/s 2021-02-20T13:07:49.575 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-02-20T13:07:49.627 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-02-20T13:07:49.628 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T13:07:49.630 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 359 Links: 1 2021-02-20T13:07:49.631 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T13:07:49.633 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2021-02-20 13:04:58.976205781 +0000 2021-02-20T13:07:49.634 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2021-02-20 13:04:58.832205754 +0000 2021-02-20T13:07:49.635 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2021-02-20 13:04:58.832205754 +0000 2021-02-20T13:07:49.635 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2021-02-20T13:07:49.636 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2021-02-20T13:07:49.637 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2021-02-20T13:07:49.637 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000283837 s, 1.8 MB/s 2021-02-20T13:07:49.640 INFO:tasks.ceph:osd dev map: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T13:07:49.640 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi170.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}} 2021-02-20T13:07:49.641 INFO:tasks.ceph:Generating config... 2021-02-20T13:07:49.644 INFO:tasks.ceph:[client] debug rbd = 20 2021-02-20T13:07:49.645 INFO:tasks.ceph:[client] debug rbd_mirror = 15 2021-02-20T13:07:49.645 INFO:tasks.ceph:[client] log to stderr = False 2021-02-20T13:07:49.646 INFO:tasks.ceph:[client] rbd default features = 125 2021-02-20T13:07:49.646 INFO:tasks.ceph:[client] rbd mirror image policy type = none 2021-02-20T13:07:49.647 INFO:tasks.ceph:[client.mirror.0] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:07:49.647 INFO:tasks.ceph:[client.mirror.0] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:07:49.648 INFO:tasks.ceph:[client.mirror.1] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:07:49.648 INFO:tasks.ceph:[client.mirror.1] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:07:49.649 INFO:tasks.ceph:[client.mirror.2] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:07:49.649 INFO:tasks.ceph:[client.mirror.2] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:07:49.650 INFO:tasks.ceph:[client.mirror.3] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:07:49.651 INFO:tasks.ceph:[client.mirror.3] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:07:49.651 INFO:tasks.ceph:[global] mon client directed command retry = 5 2021-02-20T13:07:49.652 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2021-02-20T13:07:49.652 INFO:tasks.ceph:[mgr] debug mgr = 20 2021-02-20T13:07:49.653 INFO:tasks.ceph:[mgr] debug ms = 1 2021-02-20T13:07:49.653 INFO:tasks.ceph:[mon] debug mon = 20 2021-02-20T13:07:49.654 INFO:tasks.ceph:[mon] debug ms = 1 2021-02-20T13:07:49.654 INFO:tasks.ceph:[mon] debug paxos = 20 2021-02-20T13:07:49.655 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2021-02-20T13:07:49.655 INFO:tasks.ceph:[osd] bluestore compression algorithm = lz4 2021-02-20T13:07:49.656 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2021-02-20T13:07:49.656 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2021-02-20T13:07:49.657 INFO:tasks.ceph:[osd] debug bluefs = 20 2021-02-20T13:07:49.658 INFO:tasks.ceph:[osd] debug bluestore = 20 2021-02-20T13:07:49.658 INFO:tasks.ceph:[osd] debug filestore = 20 2021-02-20T13:07:49.659 INFO:tasks.ceph:[osd] debug journal = 20 2021-02-20T13:07:49.659 INFO:tasks.ceph:[osd] debug ms = 20 2021-02-20T13:07:49.660 INFO:tasks.ceph:[osd] debug osd = 25 2021-02-20T13:07:49.661 INFO:tasks.ceph:[osd] debug rocksdb = 10 2021-02-20T13:07:49.661 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2021-02-20T13:07:49.662 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2021-02-20T13:07:49.662 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2021-02-20T13:07:49.663 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2021-02-20T13:07:49.664 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2021-02-20T13:07:49.665 INFO:tasks.ceph:Setting up cluster1.mon.a... 2021-02-20T13:07:49.665 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/cluster1.keyring 2021-02-20T13:07:49.715 INFO:teuthology.orchestra.run.smithi170.stdout:creating /etc/ceph/cluster1.keyring 2021-02-20T13:07:49.718 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/cluster1.keyring 2021-02-20T13:07:49.799 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2021-02-20T13:07:49.854 DEBUG:tasks.ceph:Ceph mon addresses: [('cluster1.mon.a', '172.21.15.170')] 2021-02-20T13:07:49.855 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.170', 'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'none'}, 'mon.a': {}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}} 2021-02-20T13:07:49.856 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:07:49.857 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2021-02-20T13:07:49.902 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.170 --print /home/ubuntu/cephtest/cluster1.monmap 2021-02-20T13:07:49.974 INFO:teuthology.orchestra.run.smithi170.stdout:monmaptool: monmap file /home/ubuntu/cephtest/cluster1.monmap 2021-02-20T13:07:49.975 INFO:teuthology.orchestra.run.smithi170.stdout:monmaptool: generated fsid b75fb0f9-8d30-4954-a231-3b62306378ca 2021-02-20T13:07:49.975 INFO:teuthology.orchestra.run.smithi170.stdout:epoch 0 2021-02-20T13:07:49.975 INFO:teuthology.orchestra.run.smithi170.stdout:fsid b75fb0f9-8d30-4954-a231-3b62306378ca 2021-02-20T13:07:49.976 INFO:teuthology.orchestra.run.smithi170.stdout:last_changed 2021-02-20T13:07:49.976449+0000 2021-02-20T13:07:49.976 INFO:teuthology.orchestra.run.smithi170.stdout:created 2021-02-20T13:07:49.976449+0000 2021-02-20T13:07:49.976 INFO:teuthology.orchestra.run.smithi170.stdout:min_mon_release 0 (unknown) 2021-02-20T13:07:49.976 INFO:teuthology.orchestra.run.smithi170.stdout:election_strategy: 1 2021-02-20T13:07:49.977 INFO:teuthology.orchestra.run.smithi170.stdout:0: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.a 2021-02-20T13:07:49.977 INFO:teuthology.orchestra.run.smithi170.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/cluster1.monmap (1 monitors) 2021-02-20T13:07:49.978 DEBUG:teuthology.orchestra.run.smithi170:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2021-02-20T13:07:50.026 INFO:tasks.ceph:Writing /etc/ceph/cluster1.conf for FSID b75fb0f9-8d30-4954-a231-3b62306378ca... 2021-02-20T13:07:50.027 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster1.conf && sudo chmod 0644 /etc/ceph/cluster1.conf > /dev/null 2021-02-20T13:07:50.070 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster1.conf && sudo chmod 0644 /etc/ceph/cluster1.conf > /dev/null 2021-02-20T13:07:50.099 INFO:teuthology.orchestra.run.smithi183.stdout:[global] 2021-02-20T13:07:50.100 INFO:teuthology.orchestra.run.smithi183.stdout: chdir = "" 2021-02-20T13:07:50.100 INFO:teuthology.orchestra.run.smithi183.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-20T13:07:50.100 INFO:teuthology.orchestra.run.smithi183.stdout: auth supported = cephx 2021-02-20T13:07:50.101 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.101 INFO:teuthology.orchestra.run.smithi183.stdout: filestore xattr use omap = true 2021-02-20T13:07:50.101 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.101 INFO:teuthology.orchestra.run.smithi183.stdout: mon clock drift allowed = 1.000 2021-02-20T13:07:50.102 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.102 INFO:teuthology.orchestra.run.smithi183.stdout: osd crush chooseleaf type = 0 2021-02-20T13:07:50.102 INFO:teuthology.orchestra.run.smithi183.stdout: auth debug = true 2021-02-20T13:07:50.102 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.103 INFO:teuthology.orchestra.run.smithi183.stdout: ms die on old message = true 2021-02-20T13:07:50.103 INFO:teuthology.orchestra.run.smithi183.stdout: ms die on bug = true 2021-02-20T13:07:50.103 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.103 INFO:teuthology.orchestra.run.smithi183.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-20T13:07:50.104 INFO:teuthology.orchestra.run.smithi183.stdout: mon pg warn max object skew = 0 2021-02-20T13:07:50.104 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.104 INFO:teuthology.orchestra.run.smithi183.stdout: # disable pg_autoscaler by default for new pools 2021-02-20T13:07:50.104 INFO:teuthology.orchestra.run.smithi183.stdout: osd_pool_default_pg_autoscale_mode = off 2021-02-20T13:07:50.104 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.105 INFO:teuthology.orchestra.run.smithi183.stdout: osd pool default size = 2 2021-02-20T13:07:50.105 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.105 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd allow primary affinity = true 2021-02-20T13:07:50.105 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd allow pg remap = true 2021-02-20T13:07:50.105 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on legacy crush tunables = false 2021-02-20T13:07:50.106 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on crush straw calc version zero = false 2021-02-20T13:07:50.106 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on no sortbitwise = false 2021-02-20T13:07:50.106 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on osd down out interval zero = false 2021-02-20T13:07:50.106 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on too few osds = false 2021-02-20T13:07:50.107 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-20T13:07:50.107 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-20T13:07:50.107 INFO:teuthology.orchestra.run.smithi183.stdout: mon_allow_pool_size_one = true 2021-02-20T13:07:50.107 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.108 INFO:teuthology.orchestra.run.smithi183.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-20T13:07:50.108 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.108 INFO:teuthology.orchestra.run.smithi183.stdout: osd default data pool replay window = 5 2021-02-20T13:07:50.108 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.109 INFO:teuthology.orchestra.run.smithi183.stdout: mon allow pool delete = true 2021-02-20T13:07:50.109 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.109 INFO:teuthology.orchestra.run.smithi183.stdout: mon cluster log file level = debug 2021-02-20T13:07:50.109 INFO:teuthology.orchestra.run.smithi183.stdout: debug asserts on shutdown = true 2021-02-20T13:07:50.109 INFO:teuthology.orchestra.run.smithi183.stdout: mon health detail to clog = false 2021-02-20T13:07:50.110 INFO:teuthology.orchestra.run.smithi183.stdout: mon host = 172.21.15.170 2021-02-20T13:07:50.110 INFO:teuthology.orchestra.run.smithi183.stdout: mon client directed command retry = 5 2021-02-20T13:07:50.118 INFO:teuthology.orchestra.run.smithi183.stdout: ms inject socket failures = 5000 2021-02-20T13:07:50.118 INFO:teuthology.orchestra.run.smithi183.stdout: fsid = b75fb0f9-8d30-4954-a231-3b62306378ca 2021-02-20T13:07:50.119 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.119 INFO:teuthology.orchestra.run.smithi183.stdout:[osd] 2021-02-20T13:07:50.119 INFO:teuthology.orchestra.run.smithi183.stdout: osd journal size = 100 2021-02-20T13:07:50.119 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.120 INFO:teuthology.orchestra.run.smithi183.stdout: osd scrub load threshold = 5.0 2021-02-20T13:07:50.120 INFO:teuthology.orchestra.run.smithi183.stdout: osd scrub max interval = 600 2021-02-20T13:07:50.120 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.120 INFO:teuthology.orchestra.run.smithi183.stdout: osd recover clone overlap = true 2021-02-20T13:07:50.120 INFO:teuthology.orchestra.run.smithi183.stdout: osd recovery max chunk = 1048576 2021-02-20T13:07:50.121 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.121 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug shutdown = true 2021-02-20T13:07:50.121 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug op order = true 2021-02-20T13:07:50.121 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug verify stray on activate = true 2021-02-20T13:07:50.122 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.122 INFO:teuthology.orchestra.run.smithi183.stdout: osd open classes on start = true 2021-02-20T13:07:50.122 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug pg log writeout = true 2021-02-20T13:07:50.122 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.122 INFO:teuthology.orchestra.run.smithi183.stdout: osd deep scrub update digest min age = 30 2021-02-20T13:07:50.123 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.123 INFO:teuthology.orchestra.run.smithi183.stdout: osd map max advance = 10 2021-02-20T13:07:50.123 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.123 INFO:teuthology.orchestra.run.smithi183.stdout: journal zero on create = true 2021-02-20T13:07:50.123 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.124 INFO:teuthology.orchestra.run.smithi183.stdout: filestore ondisk finisher threads = 3 2021-02-20T13:07:50.124 INFO:teuthology.orchestra.run.smithi183.stdout: filestore apply finisher threads = 3 2021-02-20T13:07:50.124 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.124 INFO:teuthology.orchestra.run.smithi183.stdout: bdev debug aio = true 2021-02-20T13:07:50.125 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug misdirected ops = true 2021-02-20T13:07:50.125 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore block size = 96636764160 2021-02-20T13:07:50.125 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore compression algorithm = lz4 2021-02-20T13:07:50.125 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore compression mode = aggressive 2021-02-20T13:07:50.125 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore fsck on mount = True 2021-02-20T13:07:50.126 INFO:teuthology.orchestra.run.smithi183.stdout: debug bluefs = 20 2021-02-20T13:07:50.126 INFO:teuthology.orchestra.run.smithi183.stdout: debug bluestore = 20 2021-02-20T13:07:50.126 INFO:teuthology.orchestra.run.smithi183.stdout: debug filestore = 20 2021-02-20T13:07:50.126 INFO:teuthology.orchestra.run.smithi183.stdout: debug journal = 20 2021-02-20T13:07:50.126 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 20 2021-02-20T13:07:50.127 INFO:teuthology.orchestra.run.smithi183.stdout: debug osd = 25 2021-02-20T13:07:50.127 INFO:teuthology.orchestra.run.smithi183.stdout: debug rocksdb = 10 2021-02-20T13:07:50.127 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd backfillfull_ratio = 0.85 2021-02-20T13:07:50.127 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd full ratio = 0.9 2021-02-20T13:07:50.128 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd nearfull ratio = 0.8 2021-02-20T13:07:50.128 INFO:teuthology.orchestra.run.smithi183.stdout: osd failsafe full ratio = 0.95 2021-02-20T13:07:50.128 INFO:teuthology.orchestra.run.smithi183.stdout: osd objectstore = bluestore 2021-02-20T13:07:50.128 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.129 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr] 2021-02-20T13:07:50.129 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2021-02-20T13:07:50.129 INFO:teuthology.orchestra.run.smithi183.stdout: debug mgr = 20 2021-02-20T13:07:50.129 INFO:teuthology.orchestra.run.smithi183.stdout: debug mon = 20 2021-02-20T13:07:50.146 INFO:teuthology.orchestra.run.smithi183.stdout: debug auth = 20 2021-02-20T13:07:50.146 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min pgs per osd = 4 2021-02-20T13:07:50.147 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min bytes per osd = 10 2021-02-20T13:07:50.147 INFO:teuthology.orchestra.run.smithi183.stdout: mgr/telemetry/nag = false 2021-02-20T13:07:50.147 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.147 INFO:teuthology.orchestra.run.smithi183.stdout:[mon] 2021-02-20T13:07:50.148 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2021-02-20T13:07:50.148 INFO:teuthology.orchestra.run.smithi183.stdout: debug mon = 20 2021-02-20T13:07:50.148 INFO:teuthology.orchestra.run.smithi183.stdout: debug paxos = 20 2021-02-20T13:07:50.148 INFO:teuthology.orchestra.run.smithi183.stdout: debug auth = 20 2021-02-20T13:07:50.149 INFO:teuthology.orchestra.run.smithi183.stdout: mon data avail warn = 5 2021-02-20T13:07:50.149 INFO:teuthology.orchestra.run.smithi183.stdout: mon mgr mkfs grace = 240 2021-02-20T13:07:50.149 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min pgs per osd = 4 2021-02-20T13:07:50.149 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd reporter subtree level = osd 2021-02-20T13:07:50.150 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd prime pg temp = true 2021-02-20T13:07:50.150 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min bytes per osd = 10 2021-02-20T13:07:50.150 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:07:50.150 INFO:teuthology.orchestra.run.smithi183.stdout:[client] 2021-02-20T13:07:50.150 INFO:teuthology.orchestra.run.smithi183.stdout: rgw cache enabled = true 2021-02-20T13:07:50.151 INFO:teuthology.orchestra.run.smithi183.stdout: rgw enable ops log = true 2021-02-20T13:07:50.151 INFO:teuthology.orchestra.run.smithi183.stdout: rgw enable usage log = true 2021-02-20T13:07:50.151 INFO:teuthology.orchestra.run.smithi183.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-20T13:07:50.151 INFO:teuthology.orchestra.run.smithi183.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-20T13:07:50.151 INFO:teuthology.orchestra.run.smithi183.stdout: debug rbd = 20 2021-02-20T13:07:50.152 INFO:teuthology.orchestra.run.smithi183.stdout: debug rbd_mirror = 15 2021-02-20T13:07:50.152 INFO:teuthology.orchestra.run.smithi183.stdout: log to stderr = False 2021-02-20T13:07:50.152 INFO:teuthology.orchestra.run.smithi183.stdout: rbd default features = 125 2021-02-20T13:07:50.152 INFO:teuthology.orchestra.run.smithi183.stdout: rbd mirror image policy type = none 2021-02-20T13:07:50.152 INFO:teuthology.orchestra.run.smithi183.stdout:[mon.a] 2021-02-20T13:07:50.153 INFO:teuthology.orchestra.run.smithi183.stdout:[client.mirror.0] 2021-02-20T13:07:50.153 INFO:teuthology.orchestra.run.smithi183.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:07:50.153 INFO:teuthology.orchestra.run.smithi183.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:07:50.153 INFO:teuthology.orchestra.run.smithi183.stdout:[client.mirror.1] 2021-02-20T13:07:50.153 INFO:teuthology.orchestra.run.smithi183.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:07:50.154 INFO:teuthology.orchestra.run.smithi183.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:07:50.154 INFO:teuthology.orchestra.run.smithi183.stdout:[client.mirror.2] 2021-02-20T13:07:50.154 INFO:teuthology.orchestra.run.smithi183.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:07:50.154 INFO:teuthology.orchestra.run.smithi183.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:07:50.155 INFO:teuthology.orchestra.run.smithi183.stdout:[client.mirror.3] 2021-02-20T13:07:50.155 INFO:teuthology.orchestra.run.smithi183.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:07:50.155 INFO:teuthology.orchestra.run.smithi183.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:07:50.156 INFO:teuthology.orchestra.run.smithi170.stdout:[global] 2021-02-20T13:07:50.156 INFO:teuthology.orchestra.run.smithi170.stdout: chdir = "" 2021-02-20T13:07:50.156 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-20T13:07:50.156 INFO:teuthology.orchestra.run.smithi170.stdout: auth supported = cephx 2021-02-20T13:07:50.157 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.157 INFO:teuthology.orchestra.run.smithi170.stdout: filestore xattr use omap = true 2021-02-20T13:07:50.157 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.157 INFO:teuthology.orchestra.run.smithi170.stdout: mon clock drift allowed = 1.000 2021-02-20T13:07:50.157 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.158 INFO:teuthology.orchestra.run.smithi170.stdout: osd crush chooseleaf type = 0 2021-02-20T13:07:50.158 INFO:teuthology.orchestra.run.smithi170.stdout: auth debug = true 2021-02-20T13:07:50.158 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.158 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on old message = true 2021-02-20T13:07:50.159 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on bug = true 2021-02-20T13:07:50.159 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.159 INFO:teuthology.orchestra.run.smithi170.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-20T13:07:50.159 INFO:teuthology.orchestra.run.smithi170.stdout: mon pg warn max object skew = 0 2021-02-20T13:07:50.159 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.160 INFO:teuthology.orchestra.run.smithi170.stdout: # disable pg_autoscaler by default for new pools 2021-02-20T13:07:50.160 INFO:teuthology.orchestra.run.smithi170.stdout: osd_pool_default_pg_autoscale_mode = off 2021-02-20T13:07:50.160 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.160 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default size = 2 2021-02-20T13:07:50.160 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.161 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow primary affinity = true 2021-02-20T13:07:50.161 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow pg remap = true 2021-02-20T13:07:50.161 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on legacy crush tunables = false 2021-02-20T13:07:50.161 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on crush straw calc version zero = false 2021-02-20T13:07:50.162 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on no sortbitwise = false 2021-02-20T13:07:50.162 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on osd down out interval zero = false 2021-02-20T13:07:50.176 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on too few osds = false 2021-02-20T13:07:50.176 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-20T13:07:50.177 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-20T13:07:50.177 INFO:teuthology.orchestra.run.smithi170.stdout: mon_allow_pool_size_one = true 2021-02-20T13:07:50.177 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.177 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-20T13:07:50.177 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.179 INFO:teuthology.orchestra.run.smithi170.stdout: osd default data pool replay window = 5 2021-02-20T13:07:50.179 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.179 INFO:teuthology.orchestra.run.smithi170.stdout: mon allow pool delete = true 2021-02-20T13:07:50.180 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.180 INFO:teuthology.orchestra.run.smithi170.stdout: mon cluster log file level = debug 2021-02-20T13:07:50.180 INFO:teuthology.orchestra.run.smithi170.stdout: debug asserts on shutdown = true 2021-02-20T13:07:50.181 INFO:teuthology.orchestra.run.smithi170.stdout: mon health detail to clog = false 2021-02-20T13:07:50.181 INFO:teuthology.orchestra.run.smithi170.stdout: mon host = 172.21.15.170 2021-02-20T13:07:50.181 INFO:teuthology.orchestra.run.smithi170.stdout: mon client directed command retry = 5 2021-02-20T13:07:50.181 INFO:teuthology.orchestra.run.smithi170.stdout: ms inject socket failures = 5000 2021-02-20T13:07:50.181 INFO:teuthology.orchestra.run.smithi170.stdout: fsid = b75fb0f9-8d30-4954-a231-3b62306378ca 2021-02-20T13:07:50.182 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.182 INFO:teuthology.orchestra.run.smithi170.stdout:[osd] 2021-02-20T13:07:50.182 INFO:teuthology.orchestra.run.smithi170.stdout: osd journal size = 100 2021-02-20T13:07:50.182 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.183 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub load threshold = 5.0 2021-02-20T13:07:50.183 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub max interval = 600 2021-02-20T13:07:50.183 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.185 INFO:teuthology.orchestra.run.smithi170.stdout: osd recover clone overlap = true 2021-02-20T13:07:50.187 INFO:teuthology.orchestra.run.smithi170.stdout: osd recovery max chunk = 1048576 2021-02-20T13:07:50.188 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.188 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug shutdown = true 2021-02-20T13:07:50.188 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug op order = true 2021-02-20T13:07:50.188 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug verify stray on activate = true 2021-02-20T13:07:50.188 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.189 INFO:teuthology.orchestra.run.smithi170.stdout: osd open classes on start = true 2021-02-20T13:07:50.189 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug pg log writeout = true 2021-02-20T13:07:50.189 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.189 INFO:teuthology.orchestra.run.smithi170.stdout: osd deep scrub update digest min age = 30 2021-02-20T13:07:50.190 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.190 INFO:teuthology.orchestra.run.smithi170.stdout: osd map max advance = 10 2021-02-20T13:07:50.190 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.190 INFO:teuthology.orchestra.run.smithi170.stdout: journal zero on create = true 2021-02-20T13:07:50.190 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.191 INFO:teuthology.orchestra.run.smithi170.stdout: filestore ondisk finisher threads = 3 2021-02-20T13:07:50.191 INFO:teuthology.orchestra.run.smithi170.stdout: filestore apply finisher threads = 3 2021-02-20T13:07:50.191 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.191 INFO:teuthology.orchestra.run.smithi170.stdout: bdev debug aio = true 2021-02-20T13:07:50.191 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug misdirected ops = true 2021-02-20T13:07:50.192 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore block size = 96636764160 2021-02-20T13:07:50.192 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore compression algorithm = lz4 2021-02-20T13:07:50.192 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore compression mode = aggressive 2021-02-20T13:07:50.192 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore fsck on mount = True 2021-02-20T13:07:50.193 INFO:teuthology.orchestra.run.smithi170.stdout: debug bluefs = 20 2021-02-20T13:07:50.193 INFO:teuthology.orchestra.run.smithi170.stdout: debug bluestore = 20 2021-02-20T13:07:50.193 INFO:teuthology.orchestra.run.smithi170.stdout: debug filestore = 20 2021-02-20T13:07:50.193 INFO:teuthology.orchestra.run.smithi170.stdout: debug journal = 20 2021-02-20T13:07:50.194 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 20 2021-02-20T13:07:50.194 INFO:teuthology.orchestra.run.smithi170.stdout: debug osd = 25 2021-02-20T13:07:50.194 INFO:teuthology.orchestra.run.smithi170.stdout: debug rocksdb = 10 2021-02-20T13:07:50.195 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd backfillfull_ratio = 0.85 2021-02-20T13:07:50.195 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd full ratio = 0.9 2021-02-20T13:07:50.195 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd nearfull ratio = 0.8 2021-02-20T13:07:50.195 INFO:teuthology.orchestra.run.smithi170.stdout: osd failsafe full ratio = 0.95 2021-02-20T13:07:50.196 INFO:teuthology.orchestra.run.smithi170.stdout: osd objectstore = bluestore 2021-02-20T13:07:50.196 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.196 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr] 2021-02-20T13:07:50.196 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2021-02-20T13:07:50.196 INFO:teuthology.orchestra.run.smithi170.stdout: debug mgr = 20 2021-02-20T13:07:50.197 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2021-02-20T13:07:50.197 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2021-02-20T13:07:50.198 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2021-02-20T13:07:50.198 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2021-02-20T13:07:50.198 INFO:teuthology.orchestra.run.smithi170.stdout: mgr/telemetry/nag = false 2021-02-20T13:07:50.198 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.199 INFO:teuthology.orchestra.run.smithi170.stdout:[mon] 2021-02-20T13:07:50.199 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2021-02-20T13:07:50.199 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2021-02-20T13:07:50.199 INFO:teuthology.orchestra.run.smithi170.stdout: debug paxos = 20 2021-02-20T13:07:50.199 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2021-02-20T13:07:50.200 INFO:teuthology.orchestra.run.smithi170.stdout: mon data avail warn = 5 2021-02-20T13:07:50.200 INFO:teuthology.orchestra.run.smithi170.stdout: mon mgr mkfs grace = 240 2021-02-20T13:07:50.200 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2021-02-20T13:07:50.200 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd reporter subtree level = osd 2021-02-20T13:07:50.201 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd prime pg temp = true 2021-02-20T13:07:50.201 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2021-02-20T13:07:50.201 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:07:50.201 INFO:teuthology.orchestra.run.smithi170.stdout:[client] 2021-02-20T13:07:50.201 INFO:teuthology.orchestra.run.smithi170.stdout: rgw cache enabled = true 2021-02-20T13:07:50.202 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable ops log = true 2021-02-20T13:07:50.202 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable usage log = true 2021-02-20T13:07:50.207 INFO:teuthology.orchestra.run.smithi170.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-20T13:07:50.207 INFO:teuthology.orchestra.run.smithi170.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-20T13:07:50.208 INFO:teuthology.orchestra.run.smithi170.stdout: debug rbd = 20 2021-02-20T13:07:50.208 INFO:teuthology.orchestra.run.smithi170.stdout: debug rbd_mirror = 15 2021-02-20T13:07:50.208 INFO:teuthology.orchestra.run.smithi170.stdout: log to stderr = False 2021-02-20T13:07:50.209 INFO:teuthology.orchestra.run.smithi170.stdout: rbd default features = 125 2021-02-20T13:07:50.209 INFO:teuthology.orchestra.run.smithi170.stdout: rbd mirror image policy type = none 2021-02-20T13:07:50.210 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.a] 2021-02-20T13:07:50.210 INFO:teuthology.orchestra.run.smithi170.stdout:[client.mirror.0] 2021-02-20T13:07:50.210 INFO:teuthology.orchestra.run.smithi170.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:07:50.211 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:07:50.211 INFO:teuthology.orchestra.run.smithi170.stdout:[client.mirror.1] 2021-02-20T13:07:50.212 INFO:teuthology.orchestra.run.smithi170.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:07:50.212 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:07:50.213 INFO:teuthology.orchestra.run.smithi170.stdout:[client.mirror.2] 2021-02-20T13:07:50.213 INFO:teuthology.orchestra.run.smithi170.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:07:50.214 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:07:50.215 INFO:teuthology.orchestra.run.smithi170.stdout:[client.mirror.3] 2021-02-20T13:07:50.215 INFO:teuthology.orchestra.run.smithi170.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:07:50.216 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:07:50.219 INFO:tasks.ceph:Creating admin key on cluster1.mon.a... 2021-02-20T13:07:50.220 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/cluster1.keyring 2021-02-20T13:07:50.307 INFO:tasks.ceph:Copying monmap to all nodes... 2021-02-20T13:07:50.308 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:07:50.309 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/etc/ceph/cluster1.keyring of=/dev/stdout 2021-02-20T13:07:50.324 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:07:50.324 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/home/ubuntu/cephtest/cluster1.monmap of=/dev/stdout 2021-02-20T13:07:50.374 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi170.front.sepia.ceph.com 2021-02-20T13:07:50.375 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:07:50.376 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/cluster1.keyring 2021-02-20T13:07:50.376 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2021-02-20T13:07:50.442 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:07:50.443 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/home/ubuntu/cephtest/cluster1.monmap 2021-02-20T13:07:50.491 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi183.front.sepia.ceph.com 2021-02-20T13:07:50.492 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:07:50.493 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/cluster1.keyring 2021-02-20T13:07:50.494 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2021-02-20T13:07:50.523 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:07:50.524 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/home/ubuntu/cephtest/cluster1.monmap 2021-02-20T13:07:50.570 INFO:tasks.ceph:Setting up mon nodes... 2021-02-20T13:07:50.571 INFO:tasks.ceph:Setting up mgr nodes... 2021-02-20T13:07:50.571 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mgr/cluster1-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/cluster1-x/keyring 2021-02-20T13:07:50.612 INFO:teuthology.orchestra.run.smithi170.stdout:creating /var/lib/ceph/mgr/cluster1-x/keyring 2021-02-20T13:07:50.615 INFO:tasks.ceph:Setting up mds nodes... 2021-02-20T13:07:50.615 INFO:tasks.ceph_client:Setting up client nodes... 2021-02-20T13:07:50.617 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/cluster1.client.0.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.0.keyring 2021-02-20T13:07:50.694 INFO:teuthology.orchestra.run.smithi170.stdout:creating /etc/ceph/cluster1.client.0.keyring 2021-02-20T13:07:50.706 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror /etc/ceph/cluster1.client.mirror.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.keyring 2021-02-20T13:07:50.749 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/cluster1.client.mirror.keyring 2021-02-20T13:07:50.755 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.0 /etc/ceph/cluster1.client.mirror.0.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.0.keyring 2021-02-20T13:07:50.830 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/cluster1.client.mirror.0.keyring 2021-02-20T13:07:50.845 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.1 /etc/ceph/cluster1.client.mirror.1.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.1.keyring 2021-02-20T13:07:50.923 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/cluster1.client.mirror.1.keyring 2021-02-20T13:07:50.936 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.2 /etc/ceph/cluster1.client.mirror.2.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.2.keyring 2021-02-20T13:07:51.014 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/cluster1.client.mirror.2.keyring 2021-02-20T13:07:51.024 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.3 /etc/ceph/cluster1.client.mirror.3.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.3.keyring 2021-02-20T13:07:51.100 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/cluster1.client.mirror.3.keyring 2021-02-20T13:07:51.112 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.4 /etc/ceph/cluster1.client.mirror.4.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.4.keyring 2021-02-20T13:07:51.187 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/cluster1.client.mirror.4.keyring 2021-02-20T13:07:51.200 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.5 /etc/ceph/cluster1.client.mirror.5.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.5.keyring 2021-02-20T13:07:51.290 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/cluster1.client.mirror.5.keyring 2021-02-20T13:07:51.300 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.6 /etc/ceph/cluster1.client.mirror.6.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.6.keyring 2021-02-20T13:07:51.388 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/cluster1.client.mirror.6.keyring 2021-02-20T13:07:51.419 INFO:tasks.ceph:Running mkfs on osd nodes... 2021-02-20T13:07:51.429 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi170.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}} 2021-02-20T13:07:51.430 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/cluster1-0 2021-02-20T13:07:51.458 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T13:07:51.459 INFO:tasks.ceph:role: cluster1.osd.0 2021-02-20T13:07:51.459 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi170.front.sepia.ceph.com 2021-02-20T13:07:51.460 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-02-20T13:07:51.545 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2021-02-20T13:07:51.545 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-20T13:07:51.546 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-02-20T13:07:51.546 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-02-20T13:07:51.546 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2021-02-20T13:07:51.546 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-20T13:07:51.547 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-02-20T13:07:51.547 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-20T13:07:51.547 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-20T13:07:51.548 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2021-02-20T13:07:51.548 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/cluster1-0 2021-02-20T13:07:51.617 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-0 2021-02-20T13:07:51.685 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: /sbin/restorecon: command not found 2021-02-20T13:07:51.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:07:51.688 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/cluster1-1 2021-02-20T13:07:51.759 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T13:07:51.759 INFO:tasks.ceph:role: cluster1.osd.1 2021-02-20T13:07:51.760 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi170.front.sepia.ceph.com 2021-02-20T13:07:51.760 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-02-20T13:07:51.844 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2021-02-20T13:07:51.845 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-20T13:07:51.846 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-02-20T13:07:51.846 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-02-20T13:07:51.846 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2021-02-20T13:07:51.847 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-20T13:07:51.847 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-02-20T13:07:51.848 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-20T13:07:51.848 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-20T13:07:51.852 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2021-02-20T13:07:51.853 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/cluster1-1 2021-02-20T13:07:51.940 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-1 2021-02-20T13:07:51.980 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: /sbin/restorecon: command not found 2021-02-20T13:07:51.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:07:51.982 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/cluster1-2 2021-02-20T13:07:52.039 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T13:07:52.040 INFO:tasks.ceph:role: cluster1.osd.2 2021-02-20T13:07:52.040 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi170.front.sepia.ceph.com 2021-02-20T13:07:52.040 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-02-20T13:07:52.125 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2021-02-20T13:07:52.126 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-20T13:07:52.126 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-02-20T13:07:52.127 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-02-20T13:07:52.127 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2021-02-20T13:07:52.127 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-20T13:07:52.128 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-02-20T13:07:52.128 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-20T13:07:52.129 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-20T13:07:52.139 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2021-02-20T13:07:52.139 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/cluster1-2 2021-02-20T13:07:52.198 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-2 2021-02-20T13:07:52.253 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: /sbin/restorecon: command not found 2021-02-20T13:07:52.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:07:52.255 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/cluster1.monmap 2021-02-20T13:07:52.371 INFO:teuthology.orchestra.run.smithi170.stderr:2021-02-20T13:07:52.343+0000 7fd326294080 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-0/keyring: can't open /var/lib/ceph/osd/cluster1-0/keyring: (2) No such file or directory 2021-02-20T13:07:52.372 INFO:teuthology.orchestra.run.smithi170.stderr:2021-02-20T13:07:52.343+0000 7fd326294080 -1 created new key in keyring /var/lib/ceph/osd/cluster1-0/keyring 2021-02-20T13:07:52.373 INFO:teuthology.orchestra.run.smithi170.stderr:2021-02-20T13:07:52.343+0000 7fd326294080 -1 bluestore(/var/lib/ceph/osd/cluster1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-0/block: (2) No such file or directory 2021-02-20T13:07:52.379 INFO:teuthology.orchestra.run.smithi170.stderr:2021-02-20T13:07:52.343+0000 7fd326294080 -1 bluestore(/var/lib/ceph/osd/cluster1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-0/block: (2) No such file or directory 2021-02-20T13:07:52.380 INFO:teuthology.orchestra.run.smithi170.stderr:2021-02-20T13:07:52.343+0000 7fd326294080 -1 bluestore(/var/lib/ceph/osd/cluster1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-0/block: (2) No such file or directory 2021-02-20T13:07:52.380 INFO:teuthology.orchestra.run.smithi170.stderr:2021-02-20T13:07:52.343+0000 7fd326294080 -1 bluestore(/var/lib/ceph/osd/cluster1-0) _read_fsid unparsable uuid 2021-02-20T13:07:55.309 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-0 2021-02-20T13:07:55.368 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/cluster1.monmap 2021-02-20T13:07:55.447 INFO:teuthology.orchestra.run.smithi170.stderr:2021-02-20T13:07:55.443+0000 7f1dda627080 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-1/keyring: can't open /var/lib/ceph/osd/cluster1-1/keyring: (2) No such file or directory 2021-02-20T13:07:55.447 INFO:teuthology.orchestra.run.smithi170.stderr:2021-02-20T13:07:55.443+0000 7f1dda627080 -1 created new key in keyring /var/lib/ceph/osd/cluster1-1/keyring 2021-02-20T13:07:55.447 INFO:teuthology.orchestra.run.smithi170.stderr:2021-02-20T13:07:55.443+0000 7f1dda627080 -1 bluestore(/var/lib/ceph/osd/cluster1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-1/block: (2) No such file or directory 2021-02-20T13:07:55.448 INFO:teuthology.orchestra.run.smithi170.stderr:2021-02-20T13:07:55.443+0000 7f1dda627080 -1 bluestore(/var/lib/ceph/osd/cluster1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-1/block: (2) No such file or directory 2021-02-20T13:07:55.448 INFO:teuthology.orchestra.run.smithi170.stderr:2021-02-20T13:07:55.443+0000 7f1dda627080 -1 bluestore(/var/lib/ceph/osd/cluster1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-1/block: (2) No such file or directory 2021-02-20T13:07:55.448 INFO:teuthology.orchestra.run.smithi170.stderr:2021-02-20T13:07:55.443+0000 7f1dda627080 -1 bluestore(/var/lib/ceph/osd/cluster1-1) _read_fsid unparsable uuid 2021-02-20T13:07:58.379 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-1 2021-02-20T13:07:58.435 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/cluster1.monmap 2021-02-20T13:07:58.515 INFO:teuthology.orchestra.run.smithi170.stderr:2021-02-20T13:07:58.511+0000 7f76e0876080 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-2/keyring: can't open /var/lib/ceph/osd/cluster1-2/keyring: (2) No such file or directory 2021-02-20T13:07:58.516 INFO:teuthology.orchestra.run.smithi170.stderr:2021-02-20T13:07:58.511+0000 7f76e0876080 -1 created new key in keyring /var/lib/ceph/osd/cluster1-2/keyring 2021-02-20T13:07:58.516 INFO:teuthology.orchestra.run.smithi170.stderr:2021-02-20T13:07:58.511+0000 7f76e0876080 -1 bluestore(/var/lib/ceph/osd/cluster1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-2/block: (2) No such file or directory 2021-02-20T13:07:58.516 INFO:teuthology.orchestra.run.smithi170.stderr:2021-02-20T13:07:58.511+0000 7f76e0876080 -1 bluestore(/var/lib/ceph/osd/cluster1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-2/block: (2) No such file or directory 2021-02-20T13:07:58.516 INFO:teuthology.orchestra.run.smithi170.stderr:2021-02-20T13:07:58.511+0000 7f76e0876080 -1 bluestore(/var/lib/ceph/osd/cluster1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-2/block: (2) No such file or directory 2021-02-20T13:07:58.516 INFO:teuthology.orchestra.run.smithi170.stderr:2021-02-20T13:07:58.511+0000 7f76e0876080 -1 bluestore(/var/lib/ceph/osd/cluster1-2) _read_fsid unparsable uuid 2021-02-20T13:08:01.476 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-2 2021-02-20T13:08:01.540 INFO:tasks.ceph:Reading keys from all nodes... 2021-02-20T13:08:01.541 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:08:01.541 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/mgr/cluster1-x/keyring of=/dev/stdout 2021-02-20T13:08:01.598 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:08:01.598 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/cluster1-0/keyring of=/dev/stdout 2021-02-20T13:08:01.654 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:08:01.654 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/cluster1-1/keyring of=/dev/stdout 2021-02-20T13:08:01.712 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:08:01.712 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/cluster1-2/keyring of=/dev/stdout 2021-02-20T13:08:01.777 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:08:01.778 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/etc/ceph/cluster1.client.0.keyring of=/dev/stdout 2021-02-20T13:08:01.826 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:01.826 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/cluster1.client.mirror.keyring of=/dev/stdout 2021-02-20T13:08:01.833 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:01.834 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/cluster1.client.mirror.0.keyring of=/dev/stdout 2021-02-20T13:08:01.882 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:01.882 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/cluster1.client.mirror.1.keyring of=/dev/stdout 2021-02-20T13:08:01.928 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:01.929 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/cluster1.client.mirror.2.keyring of=/dev/stdout 2021-02-20T13:08:01.976 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:01.977 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/cluster1.client.mirror.3.keyring of=/dev/stdout 2021-02-20T13:08:02.028 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:02.029 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/cluster1.client.mirror.4.keyring of=/dev/stdout 2021-02-20T13:08:02.077 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:02.078 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/cluster1.client.mirror.5.keyring of=/dev/stdout 2021-02-20T13:08:02.124 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:02.125 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/cluster1.client.mirror.6.keyring of=/dev/stdout 2021-02-20T13:08:02.176 INFO:tasks.ceph:Adding keys to all mons... 2021-02-20T13:08:02.177 DEBUG:teuthology.orchestra.run.smithi170:> sudo tee -a /etc/ceph/cluster1.keyring 2021-02-20T13:08:02.191 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr.x] 2021-02-20T13:08:02.192 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCmCTFgCKujJBAABemJ0O1tSs9S211cb5OX4Q== 2021-02-20T13:08:02.192 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.0] 2021-02-20T13:08:02.192 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCoCTFgdLrIFBAAftYy2xKacYZtt/nMk2U/7w== 2021-02-20T13:08:02.192 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.1] 2021-02-20T13:08:02.192 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCrCTFgpye5GhAAs7WQTrJd/hkHrOlTE/V6YQ== 2021-02-20T13:08:02.193 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.2] 2021-02-20T13:08:02.193 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCuCTFgQiu6HhAAYcE6J4oQHVxEF5mMK1PTjg== 2021-02-20T13:08:02.193 INFO:teuthology.orchestra.run.smithi170.stdout:[client.0] 2021-02-20T13:08:02.193 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCmCTFgwlx3KRAAZ1u29cET1bFB+elL0eUwIg== 2021-02-20T13:08:02.194 INFO:teuthology.orchestra.run.smithi170.stdout:[client.mirror] 2021-02-20T13:08:02.194 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCmCTFgfZBZLBAAuIhe3b4EPmbEPW4vjEBqsw== 2021-02-20T13:08:02.194 INFO:teuthology.orchestra.run.smithi170.stdout:[client.mirror.0] 2021-02-20T13:08:02.194 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCmCTFgBrqZMRAAqlz6Lrnwt8XmZ2lStLsOQw== 2021-02-20T13:08:02.194 INFO:teuthology.orchestra.run.smithi170.stdout:[client.mirror.1] 2021-02-20T13:08:02.195 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCmCTFg3jogNxAAgRggjD3O3rFOyOwVv35cgw== 2021-02-20T13:08:02.195 INFO:teuthology.orchestra.run.smithi170.stdout:[client.mirror.2] 2021-02-20T13:08:02.195 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCnCTFgBLzQABAA4+IrnfQEON7+ztTllpB7Yw== 2021-02-20T13:08:02.195 INFO:teuthology.orchestra.run.smithi170.stdout:[client.mirror.3] 2021-02-20T13:08:02.195 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCnCTFgzvASBhAAUwato7sRV/4lv+Gx7H319A== 2021-02-20T13:08:02.196 INFO:teuthology.orchestra.run.smithi170.stdout:[client.mirror.4] 2021-02-20T13:08:02.196 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCnCTFgw+1ECxAAF7FERID8T2qfZnGTJf/4Mw== 2021-02-20T13:08:02.196 INFO:teuthology.orchestra.run.smithi170.stdout:[client.mirror.5] 2021-02-20T13:08:02.196 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCnCTFgc0E8ERAAPMT6dhGT3RXe4NT3zPTvpA== 2021-02-20T13:08:02.197 INFO:teuthology.orchestra.run.smithi170.stdout:[client.mirror.6] 2021-02-20T13:08:02.197 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCnCTFgtNfAFhAA/8+cDHlLWx2pByLh/hZ9Nw== 2021-02-20T13:08:02.198 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-02-20T13:08:02.281 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-20T13:08:02.366 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-20T13:08:02.454 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-20T13:08:02.496 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T13:08:02.585 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T13:08:02.631 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T13:08:02.717 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T13:08:02.760 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T13:08:02.850 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T13:08:02.939 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.4 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T13:08:03.139 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.5 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T13:08:03.225 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.6 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T13:08:03.322 INFO:tasks.ceph:Running mkfs on mon nodes... 2021-02-20T13:08:03.323 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mon/cluster1-a 2021-02-20T13:08:03.378 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster cluster1 --mkfs -i a --monmap /home/ubuntu/cephtest/cluster1.monmap --keyring /etc/ceph/cluster1.keyring 2021-02-20T13:08:03.497 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/mon/cluster1-a 2021-02-20T13:08:03.514 DEBUG:teuthology.orchestra.run.smithi170:> rm -- /home/ubuntu/cephtest/cluster1.monmap 2021-02-20T13:08:03.561 INFO:tasks.ceph:Starting mon daemons in cluster cluster1... 2021-02-20T13:08:03.562 INFO:tasks.ceph.cluster1.mon.a:Restarting daemon 2021-02-20T13:08:03.564 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster cluster1 -i a 2021-02-20T13:08:03.606 INFO:tasks.ceph.cluster1.mon.a:Started 2021-02-20T13:08:03.606 INFO:tasks.ceph:Starting mgr daemons in cluster cluster1... 2021-02-20T13:08:03.607 INFO:tasks.ceph.cluster1.mgr.x:Restarting daemon 2021-02-20T13:08:03.607 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster cluster1 -i x 2021-02-20T13:08:03.609 INFO:tasks.ceph.cluster1.mgr.x:Started 2021-02-20T13:08:03.610 INFO:tasks.ceph:Setting crush tunables to default 2021-02-20T13:08:03.610 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster cluster1 osd crush tunables default 2021-02-20T13:08:04.053 INFO:teuthology.orchestra.run.smithi170.stderr:adjusted tunables profile to default 2021-02-20T13:08:04.063 INFO:tasks.ceph:Starting osd daemons in cluster cluster1... 2021-02-20T13:08:04.064 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:08:04.064 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/cluster1-0/fsid of=/dev/stdout 2021-02-20T13:08:04.079 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:08:04.079 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/cluster1-1/fsid of=/dev/stdout 2021-02-20T13:08:04.136 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:08:04.137 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/cluster1-2/fsid of=/dev/stdout 2021-02-20T13:08:04.195 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster cluster1 osd new 19af55ec-b6e8-46e9-8089-570faee45c63 0 2021-02-20T13:08:04.454 INFO:teuthology.orchestra.run.smithi170.stdout:0 2021-02-20T13:08:04.468 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster cluster1 osd new 8d8febea-7030-4d26-882e-e359adf28082 1 2021-02-20T13:08:04.696 INFO:teuthology.orchestra.run.smithi170.stdout:1 2021-02-20T13:08:04.709 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster cluster1 osd new 85a9611e-7f9d-4db8-b13d-336760a335e5 2 2021-02-20T13:08:04.932 INFO:teuthology.orchestra.run.smithi170.stdout:2 2021-02-20T13:08:04.946 INFO:tasks.ceph.cluster1.osd.0:Restarting daemon 2021-02-20T13:08:04.947 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 0 2021-02-20T13:08:04.952 INFO:tasks.ceph.cluster1.osd.0:Started 2021-02-20T13:08:04.953 INFO:tasks.ceph.cluster1.osd.1:Restarting daemon 2021-02-20T13:08:04.953 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 1 2021-02-20T13:08:04.957 INFO:tasks.ceph.cluster1.osd.1:Started 2021-02-20T13:08:04.959 INFO:tasks.ceph.cluster1.osd.2:Restarting daemon 2021-02-20T13:08:04.959 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 2 2021-02-20T13:08:04.963 INFO:tasks.ceph.cluster1.osd.2:Started 2021-02-20T13:08:04.963 INFO:tasks.ceph:Waiting for OSDs to come up 2021-02-20T13:08:05.267 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster1 osd dump --format=json 2021-02-20T13:08:05.462 INFO:teuthology.misc.health.smithi170.stdout: 2021-02-20T13:08:05.462 INFO:teuthology.misc.health.smithi170.stdout:{"epoch":5,"fsid":"b75fb0f9-8d30-4954-a231-3b62306378ca","created":"2021-02-20T13:08:03.725022+0000","modified":"2021-02-20T13:08:04.931246+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"19af55ec-b6e8-46e9-8089-570faee45c63","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8d8febea-7030-4d26-882e-e359adf28082","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"85a9611e-7f9d-4db8-b13d-336760a335e5","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T13:08:05.473 DEBUG:teuthology.misc:0 of 3 OSDs are up 2021-02-20T13:08:05.574 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T13:08:05.571+0000 7fb83c63e080 -1 Falling back to public interface 2021-02-20T13:08:05.575 INFO:tasks.ceph.cluster1.osd.1.smithi170.stderr:2021-02-20T13:08:05.571+0000 7fb11992f080 -1 Falling back to public interface 2021-02-20T13:08:05.575 INFO:tasks.ceph.cluster1.osd.0.smithi170.stderr:2021-02-20T13:08:05.571+0000 7f18631b4080 -1 Falling back to public interface 2021-02-20T13:08:09.117 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T13:08:09.115+0000 7fb83c63e080 -1 osd.2 0 log_to_monitors {default=true} 2021-02-20T13:08:09.285 INFO:tasks.ceph.cluster1.osd.0.smithi170.stderr:2021-02-20T13:08:09.283+0000 7f18631b4080 -1 osd.0 0 log_to_monitors {default=true} 2021-02-20T13:08:09.356 INFO:tasks.ceph.cluster1.osd.1.smithi170.stderr:2021-02-20T13:08:09.355+0000 7fb11992f080 -1 osd.1 0 log_to_monitors {default=true} 2021-02-20T13:08:10.951 INFO:tasks.ceph.cluster1.osd.0.smithi170.stderr:2021-02-20T13:08:10.947+0000 7f185a67d700 -1 osd.0 0 waiting for initial osdmap 2021-02-20T13:08:10.954 INFO:tasks.ceph.cluster1.osd.1.smithi170.stderr:2021-02-20T13:08:10.951+0000 7fb110df8700 -1 osd.1 0 waiting for initial osdmap 2021-02-20T13:08:10.954 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T13:08:10.951+0000 7fb833b07700 -1 osd.2 0 waiting for initial osdmap 2021-02-20T13:08:11.783 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster1 osd dump --format=json 2021-02-20T13:08:12.105 INFO:teuthology.misc.health.smithi170.stdout: 2021-02-20T13:08:12.106 INFO:teuthology.misc.health.smithi170.stdout:{"epoch":8,"fsid":"b75fb0f9-8d30-4954-a231-3b62306378ca","created":"2021-02-20T13:08:03.725022+0000","modified":"2021-02-20T13:08:11.943471+0000","last_up_change":"2021-02-20T13:08:11.943471+0000","last_in_change":"2021-02-20T13:08:11.943471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"19af55ec-b6e8-46e9-8089-570faee45c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":15761},{"type":"v1","addr":"172.21.15.170:6807","nonce":15761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6809","nonce":15761},{"type":"v1","addr":"172.21.15.170:6812","nonce":15761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":15761},{"type":"v1","addr":"172.21.15.170:6822","nonce":15761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6815","nonce":15761},{"type":"v1","addr":"172.21.15.170:6818","nonce":15761}]},"public_addr":"172.21.15.170:6807/15761","cluster_addr":"172.21.15.170:6812/15761","heartbeat_back_addr":"172.21.15.170:6822/15761","heartbeat_front_addr":"172.21.15.170:6818/15761","state":["exists","up"]},{"osd":1,"uuid":"8d8febea-7030-4d26-882e-e359adf28082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":15760},{"type":"v1","addr":"172.21.15.170:6810","nonce":15760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6813","nonce":15760},{"type":"v1","addr":"172.21.15.170:6814","nonce":15760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":15760},{"type":"v1","addr":"172.21.15.170:6823","nonce":15760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6817","nonce":15760},{"type":"v1","addr":"172.21.15.170:6819","nonce":15760}]},"public_addr":"172.21.15.170:6810/15760","cluster_addr":"172.21.15.170:6814/15760","heartbeat_back_addr":"172.21.15.170:6823/15760","heartbeat_front_addr":"172.21.15.170:6819/15760","state":["exists","up"]},{"osd":2,"uuid":"85a9611e-7f9d-4db8-b13d-336760a335e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":15762},{"type":"v1","addr":"172.21.15.170:6801","nonce":15762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":15762},{"type":"v1","addr":"172.21.15.170:6803","nonce":15762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6811","nonce":15762},{"type":"v1","addr":"172.21.15.170:6816","nonce":15762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6805","nonce":15762},{"type":"v1","addr":"172.21.15.170:6808","nonce":15762}]},"public_addr":"172.21.15.170:6801/15762","cluster_addr":"172.21.15.170:6803/15762","heartbeat_back_addr":"172.21.15.170:6816/15762","heartbeat_front_addr":"172.21.15.170:6808/15762","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T13:08:12.122 DEBUG:teuthology.misc:3 of 3 OSDs are up 2021-02-20T13:08:12.123 INFO:tasks.ceph:Creating RBD pool 2021-02-20T13:08:12.123 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster cluster1 osd pool create rbd 8 2021-02-20T13:08:12.968 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'rbd' created 2021-02-20T13:08:12.982 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster cluster1 osd pool application enable rbd rbd --yes-i-really-mean-it 2021-02-20T13:08:13.984 INFO:teuthology.orchestra.run.smithi170.stderr:enabled application 'rbd' on pool 'rbd' 2021-02-20T13:08:13.998 INFO:tasks.ceph:Starting mds daemons in cluster cluster1... 2021-02-20T13:08:13.999 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2021-02-20T13:08:14.002 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2021-02-20T13:08:14.303 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:14.304 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":10,"fsid":"b75fb0f9-8d30-4954-a231-3b62306378ca","created":"2021-02-20T13:08:03.725022+0000","modified":"2021-02-20T13:08:13.965980+0000","last_up_change":"2021-02-20T13:08:11.943471+0000","last_in_change":"2021-02-20T13:08:11.943471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-02-20T13:08:12.013434+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-02-20T13:08:12.495068+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"19af55ec-b6e8-46e9-8089-570faee45c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":15761},{"type":"v1","addr":"172.21.15.170:6807","nonce":15761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6809","nonce":15761},{"type":"v1","addr":"172.21.15.170:6812","nonce":15761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":15761},{"type":"v1","addr":"172.21.15.170:6822","nonce":15761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6815","nonce":15761},{"type":"v1","addr":"172.21.15.170:6818","nonce":15761}]},"public_addr":"172.21.15.170:6807/15761","cluster_addr":"172.21.15.170:6812/15761","heartbeat_back_addr":"172.21.15.170:6822/15761","heartbeat_front_addr":"172.21.15.170:6818/15761","state":["exists","up"]},{"osd":1,"uuid":"8d8febea-7030-4d26-882e-e359adf28082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":15760},{"type":"v1","addr":"172.21.15.170:6810","nonce":15760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6813","nonce":15760},{"type":"v1","addr":"172.21.15.170:6814","nonce":15760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":15760},{"type":"v1","addr":"172.21.15.170:6823","nonce":15760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6817","nonce":15760},{"type":"v1","addr":"172.21.15.170:6819","nonce":15760}]},"public_addr":"172.21.15.170:6810/15760","cluster_addr":"172.21.15.170:6814/15760","heartbeat_back_addr":"172.21.15.170:6823/15760","heartbeat_front_addr":"172.21.15.170:6819/15760","state":["exists","up"]},{"osd":2,"uuid":"85a9611e-7f9d-4db8-b13d-336760a335e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":15762},{"type":"v1","addr":"172.21.15.170:6801","nonce":15762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":15762},{"type":"v1","addr":"172.21.15.170:6803","nonce":15762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6811","nonce":15762},{"type":"v1","addr":"172.21.15.170:6816","nonce":15762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6805","nonce":15762},{"type":"v1","addr":"172.21.15.170:6808","nonce":15762}]},"public_addr":"172.21.15.170:6801/15762","cluster_addr":"172.21.15.170:6803/15762","heartbeat_back_addr":"172.21.15.170:6816/15762","heartbeat_front_addr":"172.21.15.170:6808/15762","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T13:08:10.282074+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T13:08:10.318134+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T13:08:10.142065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T13:08:14.318 INFO:tasks.ceph.ceph_manager.cluster1:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-02-20T13:08:12.013434+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-02-20T13:08:12.495068+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-20T13:08:14.319 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd pool get device_health_metrics pg_num 2021-02-20T13:08:14.618 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2021-02-20T13:08:14.633 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd pool get rbd pg_num 2021-02-20T13:08:14.943 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 8 2021-02-20T13:08:14.957 INFO:tasks.ceph:Waiting until cluster1 daemons up and pgs clean... 2021-02-20T13:08:14.958 INFO:tasks.ceph.ceph_manager.cluster1:waiting for mgr available 2021-02-20T13:08:14.958 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 mgr dump --format=json 2021-02-20T13:08:15.294 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:15.307 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":4,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":15577},{"type":"v1","addr":"172.21.15.170:6825","nonce":15577}]},"active_addr":"172.21.15.170:6825/15577","active_change":"2021-02-20T13:08:08.679925+0000","active_mgr_features":4540138303579357183,"available":true,"standbys":[],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"haproxy","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":217284325}]},{"addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":2345447455}]}]}} 2021-02-20T13:08:15.313 INFO:tasks.ceph.ceph_manager.cluster1:mgr available! 2021-02-20T13:08:15.313 INFO:tasks.ceph.ceph_manager.cluster1:waiting for all up 2021-02-20T13:08:15.314 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2021-02-20T13:08:15.613 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:15.613 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":11,"fsid":"b75fb0f9-8d30-4954-a231-3b62306378ca","created":"2021-02-20T13:08:03.725022+0000","modified":"2021-02-20T13:08:14.967864+0000","last_up_change":"2021-02-20T13:08:11.943471+0000","last_in_change":"2021-02-20T13:08:11.943471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-02-20T13:08:12.013434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-02-20T13:08:12.495068+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"19af55ec-b6e8-46e9-8089-570faee45c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":15761},{"type":"v1","addr":"172.21.15.170:6807","nonce":15761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6809","nonce":15761},{"type":"v1","addr":"172.21.15.170:6812","nonce":15761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":15761},{"type":"v1","addr":"172.21.15.170:6822","nonce":15761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6815","nonce":15761},{"type":"v1","addr":"172.21.15.170:6818","nonce":15761}]},"public_addr":"172.21.15.170:6807/15761","cluster_addr":"172.21.15.170:6812/15761","heartbeat_back_addr":"172.21.15.170:6822/15761","heartbeat_front_addr":"172.21.15.170:6818/15761","state":["exists","up"]},{"osd":1,"uuid":"8d8febea-7030-4d26-882e-e359adf28082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":15760},{"type":"v1","addr":"172.21.15.170:6810","nonce":15760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6813","nonce":15760},{"type":"v1","addr":"172.21.15.170:6814","nonce":15760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":15760},{"type":"v1","addr":"172.21.15.170:6823","nonce":15760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6817","nonce":15760},{"type":"v1","addr":"172.21.15.170:6819","nonce":15760}]},"public_addr":"172.21.15.170:6810/15760","cluster_addr":"172.21.15.170:6814/15760","heartbeat_back_addr":"172.21.15.170:6823/15760","heartbeat_front_addr":"172.21.15.170:6819/15760","state":["exists","up"]},{"osd":2,"uuid":"85a9611e-7f9d-4db8-b13d-336760a335e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":15762},{"type":"v1","addr":"172.21.15.170:6801","nonce":15762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":15762},{"type":"v1","addr":"172.21.15.170:6803","nonce":15762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6811","nonce":15762},{"type":"v1","addr":"172.21.15.170:6816","nonce":15762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6805","nonce":15762},{"type":"v1","addr":"172.21.15.170:6808","nonce":15762}]},"public_addr":"172.21.15.170:6801/15762","cluster_addr":"172.21.15.170:6803/15762","heartbeat_back_addr":"172.21.15.170:6816/15762","heartbeat_front_addr":"172.21.15.170:6808/15762","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T13:08:10.282074+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T13:08:10.318134+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T13:08:10.142065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T13:08:15.627 INFO:tasks.ceph.ceph_manager.cluster1:all up! 2021-02-20T13:08:15.628 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2021-02-20T13:08:15.920 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:15.920 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":11,"fsid":"b75fb0f9-8d30-4954-a231-3b62306378ca","created":"2021-02-20T13:08:03.725022+0000","modified":"2021-02-20T13:08:14.967864+0000","last_up_change":"2021-02-20T13:08:11.943471+0000","last_in_change":"2021-02-20T13:08:11.943471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-02-20T13:08:12.013434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-02-20T13:08:12.495068+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"19af55ec-b6e8-46e9-8089-570faee45c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":15761},{"type":"v1","addr":"172.21.15.170:6807","nonce":15761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6809","nonce":15761},{"type":"v1","addr":"172.21.15.170:6812","nonce":15761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":15761},{"type":"v1","addr":"172.21.15.170:6822","nonce":15761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6815","nonce":15761},{"type":"v1","addr":"172.21.15.170:6818","nonce":15761}]},"public_addr":"172.21.15.170:6807/15761","cluster_addr":"172.21.15.170:6812/15761","heartbeat_back_addr":"172.21.15.170:6822/15761","heartbeat_front_addr":"172.21.15.170:6818/15761","state":["exists","up"]},{"osd":1,"uuid":"8d8febea-7030-4d26-882e-e359adf28082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":15760},{"type":"v1","addr":"172.21.15.170:6810","nonce":15760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6813","nonce":15760},{"type":"v1","addr":"172.21.15.170:6814","nonce":15760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":15760},{"type":"v1","addr":"172.21.15.170:6823","nonce":15760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6817","nonce":15760},{"type":"v1","addr":"172.21.15.170:6819","nonce":15760}]},"public_addr":"172.21.15.170:6810/15760","cluster_addr":"172.21.15.170:6814/15760","heartbeat_back_addr":"172.21.15.170:6823/15760","heartbeat_front_addr":"172.21.15.170:6819/15760","state":["exists","up"]},{"osd":2,"uuid":"85a9611e-7f9d-4db8-b13d-336760a335e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":15762},{"type":"v1","addr":"172.21.15.170:6801","nonce":15762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":15762},{"type":"v1","addr":"172.21.15.170:6803","nonce":15762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6811","nonce":15762},{"type":"v1","addr":"172.21.15.170:6816","nonce":15762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6805","nonce":15762},{"type":"v1","addr":"172.21.15.170:6808","nonce":15762}]},"public_addr":"172.21.15.170:6801/15762","cluster_addr":"172.21.15.170:6803/15762","heartbeat_back_addr":"172.21.15.170:6816/15762","heartbeat_front_addr":"172.21.15.170:6808/15762","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T13:08:10.282074+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T13:08:10.318134+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T13:08:10.142065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T13:08:15.935 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.0 flush_pg_stats 2021-02-20T13:08:16.103 INFO:teuthology.orchestra.run.smithi170.stdout:34359738370 2021-02-20T13:08:16.104 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.1 flush_pg_stats 2021-02-20T13:08:16.286 INFO:teuthology.orchestra.run.smithi170.stdout:34359738370 2021-02-20T13:08:16.287 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.2 flush_pg_stats 2021-02-20T13:08:16.459 INFO:teuthology.orchestra.run.smithi170.stdout:34359738370 2021-02-20T13:08:16.460 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.0 2021-02-20T13:08:16.759 INFO:teuthology.orchestra.run.smithi170.stdout:34359738369 2021-02-20T13:08:16.779 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738370 got 34359738369 for osd.0 2021-02-20T13:08:17.779 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.0 2021-02-20T13:08:18.088 INFO:teuthology.orchestra.run.smithi170.stdout:34359738370 2021-02-20T13:08:18.107 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738370 got 34359738370 for osd.0 2021-02-20T13:08:18.107 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.1 2021-02-20T13:08:18.400 INFO:teuthology.orchestra.run.smithi170.stdout:34359738370 2021-02-20T13:08:18.519 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738370 got 34359738370 for osd.1 2021-02-20T13:08:18.519 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.2 2021-02-20T13:08:18.824 INFO:teuthology.orchestra.run.smithi170.stdout:34359738370 2021-02-20T13:08:18.844 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738370 got 34359738370 for osd.2 2021-02-20T13:08:18.844 INFO:tasks.ceph.ceph_manager.cluster1:waiting for clean 2021-02-20T13:08:18.844 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2021-02-20T13:08:19.142 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:19.144 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2021-02-20T13:08:19.162 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2021-02-20T13:08:18.687830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":14648,"kb_used_data":288,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":283100872,"statfs":{"total":289910292480,"available":289895292928,"internally_reserved":0,"allocated":294912,"data_stored":80052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:13.987019+0000","last_change":"2021-02-20T13:08:13.985862+0000","last_active":"2021-02-20T13:08:13.987019+0000","last_peered":"2021-02-20T13:08:13.987019+0000","last_clean":"2021-02-20T13:08:13.987019+0000","last_became_active":"2021-02-20T13:08:13.985475+0000","last_became_peered":"2021-02-20T13:08:13.985475+0000","last_unstale":"2021-02-20T13:08:13.987019+0000","last_undegraded":"2021-02-20T13:08:13.987019+0000","last_fullsized":"2021-02-20T13:08:13.987019+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_clean_scrub_stamp":"2021-02-20T13:08:12.952199+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:13.986984+0000","last_change":"2021-02-20T13:08:13.985629+0000","last_active":"2021-02-20T13:08:13.986984+0000","last_peered":"2021-02-20T13:08:13.986984+0000","last_clean":"2021-02-20T13:08:13.986984+0000","last_became_active":"2021-02-20T13:08:13.985301+0000","last_became_peered":"2021-02-20T13:08:13.985301+0000","last_unstale":"2021-02-20T13:08:13.986984+0000","last_undegraded":"2021-02-20T13:08:13.986984+0000","last_fullsized":"2021-02-20T13:08:13.986984+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_clean_scrub_stamp":"2021-02-20T13:08:12.952199+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:13.988932+0000","last_change":"2021-02-20T13:08:13.988427+0000","last_active":"2021-02-20T13:08:13.988932+0000","last_peered":"2021-02-20T13:08:13.988932+0000","last_clean":"2021-02-20T13:08:13.988932+0000","last_became_active":"2021-02-20T13:08:13.988033+0000","last_became_peered":"2021-02-20T13:08:13.988033+0000","last_unstale":"2021-02-20T13:08:13.988932+0000","last_undegraded":"2021-02-20T13:08:13.988932+0000","last_fullsized":"2021-02-20T13:08:13.988932+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_clean_scrub_stamp":"2021-02-20T13:08:12.952199+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:13.984860+0000","last_change":"2021-02-20T13:08:13.983231+0000","last_active":"2021-02-20T13:08:13.984860+0000","last_peered":"2021-02-20T13:08:13.984860+0000","last_clean":"2021-02-20T13:08:13.984860+0000","last_became_active":"2021-02-20T13:08:13.982829+0000","last_became_peered":"2021-02-20T13:08:13.982829+0000","last_unstale":"2021-02-20T13:08:13.984860+0000","last_undegraded":"2021-02-20T13:08:13.984860+0000","last_fullsized":"2021-02-20T13:08:13.984860+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_clean_scrub_stamp":"2021-02-20T13:08:12.952199+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:13.988120+0000","last_change":"2021-02-20T13:08:13.987375+0000","last_active":"2021-02-20T13:08:13.988120+0000","last_peered":"2021-02-20T13:08:13.988120+0000","last_clean":"2021-02-20T13:08:13.988120+0000","last_became_active":"2021-02-20T13:08:13.986803+0000","last_became_peered":"2021-02-20T13:08:13.986803+0000","last_unstale":"2021-02-20T13:08:13.988120+0000","last_undegraded":"2021-02-20T13:08:13.988120+0000","last_fullsized":"2021-02-20T13:08:13.988120+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_clean_scrub_stamp":"2021-02-20T13:08:12.952199+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:13.986428+0000","last_change":"2021-02-20T13:08:13.984319+0000","last_active":"2021-02-20T13:08:13.986428+0000","last_peered":"2021-02-20T13:08:13.986428+0000","last_clean":"2021-02-20T13:08:13.986428+0000","last_became_active":"2021-02-20T13:08:13.983941+0000","last_became_peered":"2021-02-20T13:08:13.983941+0000","last_unstale":"2021-02-20T13:08:13.986428+0000","last_undegraded":"2021-02-20T13:08:13.986428+0000","last_fullsized":"2021-02-20T13:08:13.986428+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_clean_scrub_stamp":"2021-02-20T13:08:12.952199+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:13.984891+0000","last_change":"2021-02-20T13:08:13.983033+0000","last_active":"2021-02-20T13:08:13.984891+0000","last_peered":"2021-02-20T13:08:13.984891+0000","last_clean":"2021-02-20T13:08:13.984891+0000","last_became_active":"2021-02-20T13:08:13.982511+0000","last_became_peered":"2021-02-20T13:08:13.982511+0000","last_unstale":"2021-02-20T13:08:13.984891+0000","last_undegraded":"2021-02-20T13:08:13.984891+0000","last_fullsized":"2021-02-20T13:08:13.984891+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_clean_scrub_stamp":"2021-02-20T13:08:12.952199+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:13.985826+0000","last_change":"2021-02-20T13:08:13.984151+0000","last_active":"2021-02-20T13:08:13.985826+0000","last_peered":"2021-02-20T13:08:13.985826+0000","last_clean":"2021-02-20T13:08:13.985826+0000","last_became_active":"2021-02-20T13:08:13.983673+0000","last_became_peered":"2021-02-20T13:08:13.983673+0000","last_unstale":"2021-02-20T13:08:13.985826+0000","last_undegraded":"2021-02-20T13:08:13.985826+0000","last_fullsized":"2021-02-20T13:08:13.985826+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_clean_scrub_stamp":"2021-02-20T13:08:12.952199+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:13.987037+0000","last_change":"2021-02-20T13:08:13.985835+0000","last_active":"2021-02-20T13:08:13.987037+0000","last_peered":"2021-02-20T13:08:13.987037+0000","last_clean":"2021-02-20T13:08:13.987037+0000","last_became_active":"2021-02-20T13:08:13.984953+0000","last_became_peered":"2021-02-20T13:08:13.984953+0000","last_unstale":"2021-02-20T13:08:13.987037+0000","last_undegraded":"2021-02-20T13:08:13.987037+0000","last_fullsized":"2021-02-20T13:08:13.987037+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_clean_scrub_stamp":"2021-02-20T13:08:12.952199+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4840,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94367000,"statfs":{"total":96636764160,"available":96631808000,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738371,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]}]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]}]}]}} 2021-02-20T13:08:19.163 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2021-02-20T13:08:19.454 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:19.457 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2021-02-20T13:08:19.469 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2021-02-20T13:08:18.687830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":14648,"kb_used_data":288,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":283100872,"statfs":{"total":289910292480,"available":289895292928,"internally_reserved":0,"allocated":294912,"data_stored":80052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:13.987019+0000","last_change":"2021-02-20T13:08:13.985862+0000","last_active":"2021-02-20T13:08:13.987019+0000","last_peered":"2021-02-20T13:08:13.987019+0000","last_clean":"2021-02-20T13:08:13.987019+0000","last_became_active":"2021-02-20T13:08:13.985475+0000","last_became_peered":"2021-02-20T13:08:13.985475+0000","last_unstale":"2021-02-20T13:08:13.987019+0000","last_undegraded":"2021-02-20T13:08:13.987019+0000","last_fullsized":"2021-02-20T13:08:13.987019+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_clean_scrub_stamp":"2021-02-20T13:08:12.952199+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:13.986984+0000","last_change":"2021-02-20T13:08:13.985629+0000","last_active":"2021-02-20T13:08:13.986984+0000","last_peered":"2021-02-20T13:08:13.986984+0000","last_clean":"2021-02-20T13:08:13.986984+0000","last_became_active":"2021-02-20T13:08:13.985301+0000","last_became_peered":"2021-02-20T13:08:13.985301+0000","last_unstale":"2021-02-20T13:08:13.986984+0000","last_undegraded":"2021-02-20T13:08:13.986984+0000","last_fullsized":"2021-02-20T13:08:13.986984+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_clean_scrub_stamp":"2021-02-20T13:08:12.952199+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:13.988932+0000","last_change":"2021-02-20T13:08:13.988427+0000","last_active":"2021-02-20T13:08:13.988932+0000","last_peered":"2021-02-20T13:08:13.988932+0000","last_clean":"2021-02-20T13:08:13.988932+0000","last_became_active":"2021-02-20T13:08:13.988033+0000","last_became_peered":"2021-02-20T13:08:13.988033+0000","last_unstale":"2021-02-20T13:08:13.988932+0000","last_undegraded":"2021-02-20T13:08:13.988932+0000","last_fullsized":"2021-02-20T13:08:13.988932+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_clean_scrub_stamp":"2021-02-20T13:08:12.952199+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:13.984860+0000","last_change":"2021-02-20T13:08:13.983231+0000","last_active":"2021-02-20T13:08:13.984860+0000","last_peered":"2021-02-20T13:08:13.984860+0000","last_clean":"2021-02-20T13:08:13.984860+0000","last_became_active":"2021-02-20T13:08:13.982829+0000","last_became_peered":"2021-02-20T13:08:13.982829+0000","last_unstale":"2021-02-20T13:08:13.984860+0000","last_undegraded":"2021-02-20T13:08:13.984860+0000","last_fullsized":"2021-02-20T13:08:13.984860+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_clean_scrub_stamp":"2021-02-20T13:08:12.952199+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:13.988120+0000","last_change":"2021-02-20T13:08:13.987375+0000","last_active":"2021-02-20T13:08:13.988120+0000","last_peered":"2021-02-20T13:08:13.988120+0000","last_clean":"2021-02-20T13:08:13.988120+0000","last_became_active":"2021-02-20T13:08:13.986803+0000","last_became_peered":"2021-02-20T13:08:13.986803+0000","last_unstale":"2021-02-20T13:08:13.988120+0000","last_undegraded":"2021-02-20T13:08:13.988120+0000","last_fullsized":"2021-02-20T13:08:13.988120+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_clean_scrub_stamp":"2021-02-20T13:08:12.952199+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:13.986428+0000","last_change":"2021-02-20T13:08:13.984319+0000","last_active":"2021-02-20T13:08:13.986428+0000","last_peered":"2021-02-20T13:08:13.986428+0000","last_clean":"2021-02-20T13:08:13.986428+0000","last_became_active":"2021-02-20T13:08:13.983941+0000","last_became_peered":"2021-02-20T13:08:13.983941+0000","last_unstale":"2021-02-20T13:08:13.986428+0000","last_undegraded":"2021-02-20T13:08:13.986428+0000","last_fullsized":"2021-02-20T13:08:13.986428+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_clean_scrub_stamp":"2021-02-20T13:08:12.952199+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:13.984891+0000","last_change":"2021-02-20T13:08:13.983033+0000","last_active":"2021-02-20T13:08:13.984891+0000","last_peered":"2021-02-20T13:08:13.984891+0000","last_clean":"2021-02-20T13:08:13.984891+0000","last_became_active":"2021-02-20T13:08:13.982511+0000","last_became_peered":"2021-02-20T13:08:13.982511+0000","last_unstale":"2021-02-20T13:08:13.984891+0000","last_undegraded":"2021-02-20T13:08:13.984891+0000","last_fullsized":"2021-02-20T13:08:13.984891+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_clean_scrub_stamp":"2021-02-20T13:08:12.952199+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:13.985826+0000","last_change":"2021-02-20T13:08:13.984151+0000","last_active":"2021-02-20T13:08:13.985826+0000","last_peered":"2021-02-20T13:08:13.985826+0000","last_clean":"2021-02-20T13:08:13.985826+0000","last_became_active":"2021-02-20T13:08:13.983673+0000","last_became_peered":"2021-02-20T13:08:13.983673+0000","last_unstale":"2021-02-20T13:08:13.985826+0000","last_undegraded":"2021-02-20T13:08:13.985826+0000","last_fullsized":"2021-02-20T13:08:13.985826+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_clean_scrub_stamp":"2021-02-20T13:08:12.952199+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:13.987037+0000","last_change":"2021-02-20T13:08:13.985835+0000","last_active":"2021-02-20T13:08:13.987037+0000","last_peered":"2021-02-20T13:08:13.987037+0000","last_clean":"2021-02-20T13:08:13.987037+0000","last_became_active":"2021-02-20T13:08:13.984953+0000","last_became_peered":"2021-02-20T13:08:13.984953+0000","last_unstale":"2021-02-20T13:08:13.987037+0000","last_undegraded":"2021-02-20T13:08:13.987037+0000","last_fullsized":"2021-02-20T13:08:13.987037+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:12.952199+0000","last_clean_scrub_stamp":"2021-02-20T13:08:12.952199+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4840,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94367000,"statfs":{"total":96636764160,"available":96631808000,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738371,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]}]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]}]}]}} 2021-02-20T13:08:19.470 INFO:tasks.ceph.ceph_manager.cluster1:clean! 2021-02-20T13:08:19.471 INFO:tasks.ceph:Waiting until ceph cluster cluster1 is healthy... 2021-02-20T13:08:19.471 INFO:tasks.ceph.ceph_manager.cluster1:wait_until_healthy 2021-02-20T13:08:19.472 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 health --format=json 2021-02-20T13:08:19.815 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:19.815 INFO:teuthology.orchestra.run.smithi170.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2021-02-20T13:08:19.829 INFO:tasks.ceph.ceph_manager.cluster1:wait_until_healthy done 2021-02-20T13:08:19.830 INFO:teuthology.run_tasks:Running task ceph... 2021-02-20T13:08:19.844 INFO:tasks.ceph:Creating ceph cluster cluster2... 2021-02-20T13:08:19.845 INFO:tasks.ceph:config {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'none'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(OSD_SLOW_PING_TIME'], 'cpu_profile': set(), 'cluster': 'cluster2', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2021-02-20T13:08:19.847 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/trociny-2021-02-20_12:31:02-rbd-wip-mgolub-testing-distro-basic-smithi/5898608', 'branch': 'wip-mgolub-testing', 'description': 'rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}', 'email': 'to.my.trociny@gmail.com', 'first_in_suite': False, 'job_id': '5898608', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': 'run two ceph clusters and install rbd-mirror'}, {'desc': 'configure the permissions for client.mirror'}, {'desc': '2 ceph clusters with 1 mon and 3 osds each'}, {'desc': 'run four rbd-mirror daemons per cluster'}, {'desc': 'run the rbd_mirror_stress.sh workunit to test the rbd-mirror daemon'}], 'name': 'trociny-2021-02-20_12:31:02-rbd-wip-mgolub-testing-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '18.04', 'overrides': {'admin_socket': {'branch': 'wip-mgolub-testing'}, 'ceph': {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'none'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(OSD_SLOW_PING_TIME'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '0d796331675b92762d821b40e70f6598bcea1178'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': '0d796331675b92762d821b40e70f6598bcea1178'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mgolub-testing', 'sha1': '0d796331675b92762d821b40e70f6598bcea1178'}}, 'owner': 'scheduled_trociny@teuthology', 'priority': 50, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'], ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']], 'sha1': '0d796331675b92762d821b40e70f6598bcea1178', 'sleep_before_teardown': 0, 'suite': 'rbd', 'suite_branch': 'wip-mgolub-testing', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_0d796331675b92762d821b40e70f6598bcea1178/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '0d796331675b92762d821b40e70f6598bcea1178', 'targets': {'smithi170.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDfrY9y6okoGGmSdGP0lWbQ4ocf5wj8XLxLWzIzKeOEY6RLGsDGfOqyKmBHiiztSFrBhM9jjTJxOvvPlTSrU9d2Dgm1QmuEjYA1JLPSaO8piMG/OlaSjI4PHxt9nxvTu77gXoXfXpK6Qo+ZUqZRv3OF9nApXNfTuDTYec8solgHl1hrUrW+n+Ow9gmRFNrBkDqVvEnYG7Xg+EQEm5aelmrdIhIdLh5Mjum14tIxk4i/1a2umeJOt88yzRzjLS2+H2/EXUn5QOGGpkz6mCS1Xd5L9UP4JVeYG9wdykGBhPJykA6OCzH5JEbGUEvqoSruNpN65AWm8PTlQ9hYB4OoK0VX', 'smithi183.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC7bsnX0wXavEUXQ1xu3LEdpYndhq6jmv7ggYLSbXPw77TjZBIbrBrtxwh0XeWqiP9d2H0n0hf/drGz5LjqiplvzH9leTtkoAE3Y+0niWLqunqLFKlAqLRKPSrZKtJtc0kDrn34lpO5aORxGAD/ZfqsRrXLLNYgD3VV9XLNPPldmEFTjPOXJa1R2mAhDDfEFXuj/pZxhL+LlJytgvOI6EGcrYgixGtDVoQ2b8zbsz/4Z/0YN/TvhEE+TZs22LKfPiivEoAMv21FVua6WDv9WX526ChOhnkLnBFJXRavZG4r14sd4+2aWAgd+cS47CDYTuJQxs2hdzY8crT3ZRJPsIRL'}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': ['rbd-mirror'], 'sha1': '0d796331675b92762d821b40e70f6598bcea1178'}}, {'ceph': {'cluster': 'cluster1', 'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'none'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(OSD_SLOW_PING_TIME'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '0d796331675b92762d821b40e70f6598bcea1178'}}, {'ceph': {'cluster': 'cluster2', 'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'none'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(OSD_SLOW_PING_TIME'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '0d796331675b92762d821b40e70f6598bcea1178'}}, {'exec': {'cluster1.client.mirror.0': ["sudo ceph --cluster cluster1 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'"]}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.3', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.3', 'thrash': True}}, {'rbd-mirror-thrash': {'cluster': 'cluster1'}}, {'rbd-mirror-thrash': {'cluster': 'cluster2'}}, {'workunit': {'clients': {'cluster1.client.mirror': ['rbd/rbd_mirror_stress.sh']}, 'env': {'CEPH_ARGS': '', 'MIRROR_IMAGE_MODE': 'snapshot', 'MIRROR_POOL_MODE': 'image', 'RBD_IMAGE_FEATURES': 'layering,exclusive-lock', 'RBD_MIRROR_INSTANCES': '4', 'RBD_MIRROR_USE_EXISTING_CLUSTER': '1', 'RBD_MIRROR_USE_RBD_MIRROR': '1'}, 'timeout': '6h'}}], 'teuthology_branch': 'master', 'tube': 'smithi', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.29353'} 2021-02-20T13:08:19.849 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/cluster2.data 2021-02-20T13:08:19.856 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/cluster2.data 2021-02-20T13:08:19.864 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m0777 -- /var/run/ceph 2021-02-20T13:08:19.901 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m0777 -- /var/run/ceph 2021-02-20T13:08:19.922 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:19.923 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2021-02-20T13:08:19.972 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-02-20T13:08:19.972 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-02-20T13:08:20.021 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-02-20T13:08:20.022 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T13:08:20.022 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 6h/6d Inode: 340 Links: 1 2021-02-20T13:08:20.022 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T13:08:20.022 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2021-02-20 13:04:58.436618145 +0000 2021-02-20T13:08:20.022 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2021-02-20 13:04:58.344664133 +0000 2021-02-20T13:08:20.023 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2021-02-20 13:04:58.344664133 +0000 2021-02-20T13:08:20.023 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2021-02-20T13:08:20.031 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2021-02-20T13:08:20.032 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2021-02-20T13:08:20.032 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000397424 s, 1.3 MB/s 2021-02-20T13:08:20.036 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-02-20T13:08:20.085 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-02-20T13:08:20.085 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T13:08:20.085 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 6h/6d Inode: 346 Links: 1 2021-02-20T13:08:20.086 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T13:08:20.086 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2021-02-20 13:04:58.724474184 +0000 2021-02-20T13:08:20.086 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2021-02-20 13:04:58.640516173 +0000 2021-02-20T13:08:20.086 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2021-02-20 13:04:58.640516173 +0000 2021-02-20T13:08:20.086 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2021-02-20T13:08:20.094 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2021-02-20T13:08:20.094 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2021-02-20T13:08:20.094 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000348311 s, 1.5 MB/s 2021-02-20T13:08:20.097 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-02-20T13:08:20.144 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-02-20T13:08:20.145 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T13:08:20.145 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 6h/6d Inode: 352 Links: 1 2021-02-20T13:08:20.146 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T13:08:20.146 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2021-02-20 13:04:58.848412200 +0000 2021-02-20T13:08:20.146 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2021-02-20 13:04:58.844414200 +0000 2021-02-20T13:08:20.147 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2021-02-20 13:04:58.844414200 +0000 2021-02-20T13:08:20.147 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2021-02-20T13:08:20.155 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2021-02-20T13:08:20.156 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2021-02-20T13:08:20.157 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000381413 s, 1.3 MB/s 2021-02-20T13:08:20.161 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-02-20T13:08:20.209 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-02-20T13:08:20.210 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T13:08:20.211 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 6h/6d Inode: 358 Links: 1 2021-02-20T13:08:20.211 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T13:08:20.212 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2021-02-20 13:04:59.060306229 +0000 2021-02-20T13:08:20.212 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2021-02-20 13:04:59.056308228 +0000 2021-02-20T13:08:20.215 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2021-02-20 13:04:59.056308228 +0000 2021-02-20T13:08:20.215 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2021-02-20T13:08:20.218 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2021-02-20T13:08:20.218 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2021-02-20T13:08:20.219 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000355958 s, 1.4 MB/s 2021-02-20T13:08:20.223 INFO:tasks.ceph:osd dev map: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T13:08:20.224 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi183.front.sepia.ceph.com'): {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'}} 2021-02-20T13:08:20.226 INFO:tasks.ceph:Generating config... 2021-02-20T13:08:20.229 INFO:tasks.ceph:[client] debug rbd = 20 2021-02-20T13:08:20.230 INFO:tasks.ceph:[client] debug rbd_mirror = 15 2021-02-20T13:08:20.230 INFO:tasks.ceph:[client] log to stderr = False 2021-02-20T13:08:20.230 INFO:tasks.ceph:[client] rbd default features = 125 2021-02-20T13:08:20.231 INFO:tasks.ceph:[client] rbd mirror image policy type = none 2021-02-20T13:08:20.231 INFO:tasks.ceph:[client.mirror.0] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:08:20.231 INFO:tasks.ceph:[client.mirror.0] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:08:20.232 INFO:tasks.ceph:[client.mirror.1] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:08:20.232 INFO:tasks.ceph:[client.mirror.1] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:08:20.232 INFO:tasks.ceph:[client.mirror.2] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:08:20.232 INFO:tasks.ceph:[client.mirror.2] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:08:20.233 INFO:tasks.ceph:[client.mirror.3] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:08:20.233 INFO:tasks.ceph:[client.mirror.3] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:08:20.233 INFO:tasks.ceph:[global] mon client directed command retry = 5 2021-02-20T13:08:20.234 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2021-02-20T13:08:20.234 INFO:tasks.ceph:[mgr] debug mgr = 20 2021-02-20T13:08:20.234 INFO:tasks.ceph:[mgr] debug ms = 1 2021-02-20T13:08:20.234 INFO:tasks.ceph:[mon] debug mon = 20 2021-02-20T13:08:20.235 INFO:tasks.ceph:[mon] debug ms = 1 2021-02-20T13:08:20.235 INFO:tasks.ceph:[mon] debug paxos = 20 2021-02-20T13:08:20.235 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2021-02-20T13:08:20.236 INFO:tasks.ceph:[osd] bluestore compression algorithm = lz4 2021-02-20T13:08:20.236 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2021-02-20T13:08:20.236 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2021-02-20T13:08:20.237 INFO:tasks.ceph:[osd] debug bluefs = 20 2021-02-20T13:08:20.237 INFO:tasks.ceph:[osd] debug bluestore = 20 2021-02-20T13:08:20.237 INFO:tasks.ceph:[osd] debug filestore = 20 2021-02-20T13:08:20.238 INFO:tasks.ceph:[osd] debug journal = 20 2021-02-20T13:08:20.238 INFO:tasks.ceph:[osd] debug ms = 20 2021-02-20T13:08:20.238 INFO:tasks.ceph:[osd] debug osd = 25 2021-02-20T13:08:20.239 INFO:tasks.ceph:[osd] debug rocksdb = 10 2021-02-20T13:08:20.239 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2021-02-20T13:08:20.239 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2021-02-20T13:08:20.240 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2021-02-20T13:08:20.240 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2021-02-20T13:08:20.240 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2021-02-20T13:08:20.241 INFO:tasks.ceph:Setting up cluster2.mon.a... 2021-02-20T13:08:20.241 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/cluster2.keyring 2021-02-20T13:08:20.298 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/cluster2.keyring 2021-02-20T13:08:20.300 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/cluster2.keyring 2021-02-20T13:08:20.375 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2021-02-20T13:08:20.432 DEBUG:tasks.ceph:Ceph mon addresses: [('cluster2.mon.a', '172.21.15.183')] 2021-02-20T13:08:20.433 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.183', 'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'none'}, 'mon.a': {}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}} 2021-02-20T13:08:20.435 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:20.435 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2021-02-20T13:08:20.480 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.183 --print /home/ubuntu/cephtest/cluster2.monmap 2021-02-20T13:08:20.554 INFO:teuthology.orchestra.run.smithi183.stdout:monmaptool: monmap file /home/ubuntu/cephtest/cluster2.monmap 2021-02-20T13:08:20.554 INFO:teuthology.orchestra.run.smithi183.stdout:monmaptool: generated fsid f8fe48d9-618e-4952-943d-8a8e6ffe912c 2021-02-20T13:08:20.555 INFO:teuthology.orchestra.run.smithi183.stdout:epoch 0 2021-02-20T13:08:20.555 INFO:teuthology.orchestra.run.smithi183.stdout:fsid f8fe48d9-618e-4952-943d-8a8e6ffe912c 2021-02-20T13:08:20.555 INFO:teuthology.orchestra.run.smithi183.stdout:last_changed 2021-02-20T13:08:20.556882+0000 2021-02-20T13:08:20.556 INFO:teuthology.orchestra.run.smithi183.stdout:created 2021-02-20T13:08:20.556882+0000 2021-02-20T13:08:20.556 INFO:teuthology.orchestra.run.smithi183.stdout:min_mon_release 0 (unknown) 2021-02-20T13:08:20.556 INFO:teuthology.orchestra.run.smithi183.stdout:election_strategy: 1 2021-02-20T13:08:20.556 INFO:teuthology.orchestra.run.smithi183.stdout:0: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.a 2021-02-20T13:08:20.556 INFO:teuthology.orchestra.run.smithi183.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/cluster2.monmap (1 monitors) 2021-02-20T13:08:20.558 DEBUG:teuthology.orchestra.run.smithi183:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2021-02-20T13:08:20.604 INFO:tasks.ceph:Writing /etc/ceph/cluster2.conf for FSID f8fe48d9-618e-4952-943d-8a8e6ffe912c... 2021-02-20T13:08:20.606 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster2.conf && sudo chmod 0644 /etc/ceph/cluster2.conf > /dev/null 2021-02-20T13:08:20.608 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster2.conf && sudo chmod 0644 /etc/ceph/cluster2.conf > /dev/null 2021-02-20T13:08:20.649 INFO:teuthology.orchestra.run.smithi170.stdout:[global] 2021-02-20T13:08:20.649 INFO:teuthology.orchestra.run.smithi170.stdout: chdir = "" 2021-02-20T13:08:20.650 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-20T13:08:20.650 INFO:teuthology.orchestra.run.smithi170.stdout: auth supported = cephx 2021-02-20T13:08:20.650 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.650 INFO:teuthology.orchestra.run.smithi170.stdout: filestore xattr use omap = true 2021-02-20T13:08:20.651 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.651 INFO:teuthology.orchestra.run.smithi170.stdout: mon clock drift allowed = 1.000 2021-02-20T13:08:20.651 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.652 INFO:teuthology.orchestra.run.smithi170.stdout: osd crush chooseleaf type = 0 2021-02-20T13:08:20.652 INFO:teuthology.orchestra.run.smithi170.stdout: auth debug = true 2021-02-20T13:08:20.652 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.652 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on old message = true 2021-02-20T13:08:20.653 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on bug = true 2021-02-20T13:08:20.653 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.653 INFO:teuthology.orchestra.run.smithi170.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-20T13:08:20.653 INFO:teuthology.orchestra.run.smithi170.stdout: mon pg warn max object skew = 0 2021-02-20T13:08:20.654 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.654 INFO:teuthology.orchestra.run.smithi170.stdout: # disable pg_autoscaler by default for new pools 2021-02-20T13:08:20.654 INFO:teuthology.orchestra.run.smithi170.stdout: osd_pool_default_pg_autoscale_mode = off 2021-02-20T13:08:20.654 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.655 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default size = 2 2021-02-20T13:08:20.655 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.655 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow primary affinity = true 2021-02-20T13:08:20.655 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow pg remap = true 2021-02-20T13:08:20.656 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on legacy crush tunables = false 2021-02-20T13:08:20.656 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on crush straw calc version zero = false 2021-02-20T13:08:20.656 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on no sortbitwise = false 2021-02-20T13:08:20.657 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on osd down out interval zero = false 2021-02-20T13:08:20.657 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on too few osds = false 2021-02-20T13:08:20.657 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-20T13:08:20.657 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-20T13:08:20.658 INFO:teuthology.orchestra.run.smithi170.stdout: mon_allow_pool_size_one = true 2021-02-20T13:08:20.658 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.658 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-20T13:08:20.658 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.659 INFO:teuthology.orchestra.run.smithi170.stdout: osd default data pool replay window = 5 2021-02-20T13:08:20.659 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.659 INFO:teuthology.orchestra.run.smithi170.stdout: mon allow pool delete = true 2021-02-20T13:08:20.659 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.660 INFO:teuthology.orchestra.run.smithi170.stdout: mon cluster log file level = debug 2021-02-20T13:08:20.660 INFO:teuthology.orchestra.run.smithi170.stdout: debug asserts on shutdown = true 2021-02-20T13:08:20.660 INFO:teuthology.orchestra.run.smithi170.stdout: mon health detail to clog = false 2021-02-20T13:08:20.660 INFO:teuthology.orchestra.run.smithi170.stdout: mon host = 172.21.15.183 2021-02-20T13:08:20.661 INFO:teuthology.orchestra.run.smithi170.stdout: mon client directed command retry = 5 2021-02-20T13:08:20.661 INFO:teuthology.orchestra.run.smithi170.stdout: ms inject socket failures = 5000 2021-02-20T13:08:20.661 INFO:teuthology.orchestra.run.smithi170.stdout: fsid = f8fe48d9-618e-4952-943d-8a8e6ffe912c 2021-02-20T13:08:20.661 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.661 INFO:teuthology.orchestra.run.smithi170.stdout:[osd] 2021-02-20T13:08:20.662 INFO:teuthology.orchestra.run.smithi170.stdout: osd journal size = 100 2021-02-20T13:08:20.677 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.677 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub load threshold = 5.0 2021-02-20T13:08:20.677 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub max interval = 600 2021-02-20T13:08:20.678 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.678 INFO:teuthology.orchestra.run.smithi170.stdout: osd recover clone overlap = true 2021-02-20T13:08:20.678 INFO:teuthology.orchestra.run.smithi170.stdout: osd recovery max chunk = 1048576 2021-02-20T13:08:20.679 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.679 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug shutdown = true 2021-02-20T13:08:20.679 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug op order = true 2021-02-20T13:08:20.679 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug verify stray on activate = true 2021-02-20T13:08:20.680 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.680 INFO:teuthology.orchestra.run.smithi170.stdout: osd open classes on start = true 2021-02-20T13:08:20.680 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug pg log writeout = true 2021-02-20T13:08:20.680 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.681 INFO:teuthology.orchestra.run.smithi170.stdout: osd deep scrub update digest min age = 30 2021-02-20T13:08:20.681 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.681 INFO:teuthology.orchestra.run.smithi170.stdout: osd map max advance = 10 2021-02-20T13:08:20.681 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.682 INFO:teuthology.orchestra.run.smithi170.stdout: journal zero on create = true 2021-02-20T13:08:20.682 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.682 INFO:teuthology.orchestra.run.smithi170.stdout: filestore ondisk finisher threads = 3 2021-02-20T13:08:20.682 INFO:teuthology.orchestra.run.smithi170.stdout: filestore apply finisher threads = 3 2021-02-20T13:08:20.683 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.683 INFO:teuthology.orchestra.run.smithi170.stdout: bdev debug aio = true 2021-02-20T13:08:20.683 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug misdirected ops = true 2021-02-20T13:08:20.683 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore block size = 96636764160 2021-02-20T13:08:20.683 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore compression algorithm = lz4 2021-02-20T13:08:20.684 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore compression mode = aggressive 2021-02-20T13:08:20.684 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore fsck on mount = True 2021-02-20T13:08:20.684 INFO:teuthology.orchestra.run.smithi170.stdout: debug bluefs = 20 2021-02-20T13:08:20.684 INFO:teuthology.orchestra.run.smithi170.stdout: debug bluestore = 20 2021-02-20T13:08:20.685 INFO:teuthology.orchestra.run.smithi170.stdout: debug filestore = 20 2021-02-20T13:08:20.685 INFO:teuthology.orchestra.run.smithi170.stdout: debug journal = 20 2021-02-20T13:08:20.685 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 20 2021-02-20T13:08:20.685 INFO:teuthology.orchestra.run.smithi170.stdout: debug osd = 25 2021-02-20T13:08:20.685 INFO:teuthology.orchestra.run.smithi170.stdout: debug rocksdb = 10 2021-02-20T13:08:20.686 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd backfillfull_ratio = 0.85 2021-02-20T13:08:20.686 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd full ratio = 0.9 2021-02-20T13:08:20.686 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd nearfull ratio = 0.8 2021-02-20T13:08:20.686 INFO:teuthology.orchestra.run.smithi170.stdout: osd failsafe full ratio = 0.95 2021-02-20T13:08:20.686 INFO:teuthology.orchestra.run.smithi170.stdout: osd objectstore = bluestore 2021-02-20T13:08:20.687 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.688 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr] 2021-02-20T13:08:20.688 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2021-02-20T13:08:20.689 INFO:teuthology.orchestra.run.smithi170.stdout: debug mgr = 20 2021-02-20T13:08:20.689 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2021-02-20T13:08:20.690 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2021-02-20T13:08:20.690 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2021-02-20T13:08:20.690 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2021-02-20T13:08:20.691 INFO:teuthology.orchestra.run.smithi170.stdout: mgr/telemetry/nag = false 2021-02-20T13:08:20.691 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.691 INFO:teuthology.orchestra.run.smithi170.stdout:[mon] 2021-02-20T13:08:20.691 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2021-02-20T13:08:20.691 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2021-02-20T13:08:20.692 INFO:teuthology.orchestra.run.smithi170.stdout: debug paxos = 20 2021-02-20T13:08:20.692 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2021-02-20T13:08:20.692 INFO:teuthology.orchestra.run.smithi170.stdout: mon data avail warn = 5 2021-02-20T13:08:20.692 INFO:teuthology.orchestra.run.smithi170.stdout: mon mgr mkfs grace = 240 2021-02-20T13:08:20.692 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2021-02-20T13:08:20.693 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd reporter subtree level = osd 2021-02-20T13:08:20.693 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd prime pg temp = true 2021-02-20T13:08:20.693 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2021-02-20T13:08:20.693 INFO:teuthology.orchestra.run.smithi170.stdout: 2021-02-20T13:08:20.693 INFO:teuthology.orchestra.run.smithi170.stdout:[client] 2021-02-20T13:08:20.694 INFO:teuthology.orchestra.run.smithi170.stdout: rgw cache enabled = true 2021-02-20T13:08:20.694 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable ops log = true 2021-02-20T13:08:20.694 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable usage log = true 2021-02-20T13:08:20.694 INFO:teuthology.orchestra.run.smithi170.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-20T13:08:20.695 INFO:teuthology.orchestra.run.smithi170.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-20T13:08:20.695 INFO:teuthology.orchestra.run.smithi170.stdout: debug rbd = 20 2021-02-20T13:08:20.695 INFO:teuthology.orchestra.run.smithi170.stdout: debug rbd_mirror = 15 2021-02-20T13:08:20.695 INFO:teuthology.orchestra.run.smithi170.stdout: log to stderr = False 2021-02-20T13:08:20.695 INFO:teuthology.orchestra.run.smithi170.stdout: rbd default features = 125 2021-02-20T13:08:20.696 INFO:teuthology.orchestra.run.smithi170.stdout: rbd mirror image policy type = none 2021-02-20T13:08:20.696 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.a] 2021-02-20T13:08:20.696 INFO:teuthology.orchestra.run.smithi170.stdout:[client.mirror.0] 2021-02-20T13:08:20.696 INFO:teuthology.orchestra.run.smithi170.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:08:20.697 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:08:20.697 INFO:teuthology.orchestra.run.smithi170.stdout:[client.mirror.1] 2021-02-20T13:08:20.697 INFO:teuthology.orchestra.run.smithi170.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:08:20.697 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:08:20.698 INFO:teuthology.orchestra.run.smithi170.stdout:[client.mirror.2] 2021-02-20T13:08:20.698 INFO:teuthology.orchestra.run.smithi170.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:08:20.698 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:08:20.698 INFO:teuthology.orchestra.run.smithi170.stdout:[client.mirror.3] 2021-02-20T13:08:20.698 INFO:teuthology.orchestra.run.smithi170.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:08:20.699 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:08:20.700 INFO:teuthology.orchestra.run.smithi183.stdout:[global] 2021-02-20T13:08:20.700 INFO:teuthology.orchestra.run.smithi183.stdout: chdir = "" 2021-02-20T13:08:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-20T13:08:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: auth supported = cephx 2021-02-20T13:08:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: filestore xattr use omap = true 2021-02-20T13:08:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: mon clock drift allowed = 1.000 2021-02-20T13:08:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: osd crush chooseleaf type = 0 2021-02-20T13:08:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: auth debug = true 2021-02-20T13:08:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: ms die on old message = true 2021-02-20T13:08:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: ms die on bug = true 2021-02-20T13:08:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-20T13:08:20.704 INFO:teuthology.orchestra.run.smithi183.stdout: mon pg warn max object skew = 0 2021-02-20T13:08:20.704 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.704 INFO:teuthology.orchestra.run.smithi183.stdout: # disable pg_autoscaler by default for new pools 2021-02-20T13:08:20.704 INFO:teuthology.orchestra.run.smithi183.stdout: osd_pool_default_pg_autoscale_mode = off 2021-02-20T13:08:20.704 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: osd pool default size = 2 2021-02-20T13:08:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd allow primary affinity = true 2021-02-20T13:08:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd allow pg remap = true 2021-02-20T13:08:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on legacy crush tunables = false 2021-02-20T13:08:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on crush straw calc version zero = false 2021-02-20T13:08:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on no sortbitwise = false 2021-02-20T13:08:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on osd down out interval zero = false 2021-02-20T13:08:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on too few osds = false 2021-02-20T13:08:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-20T13:08:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-20T13:08:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: mon_allow_pool_size_one = true 2021-02-20T13:08:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-20T13:08:20.708 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.708 INFO:teuthology.orchestra.run.smithi183.stdout: osd default data pool replay window = 5 2021-02-20T13:08:20.708 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.708 INFO:teuthology.orchestra.run.smithi183.stdout: mon allow pool delete = true 2021-02-20T13:08:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: mon cluster log file level = debug 2021-02-20T13:08:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: debug asserts on shutdown = true 2021-02-20T13:08:20.710 INFO:teuthology.orchestra.run.smithi183.stdout: mon health detail to clog = false 2021-02-20T13:08:20.710 INFO:teuthology.orchestra.run.smithi183.stdout: mon host = 172.21.15.183 2021-02-20T13:08:20.710 INFO:teuthology.orchestra.run.smithi183.stdout: mon client directed command retry = 5 2021-02-20T13:08:20.710 INFO:teuthology.orchestra.run.smithi183.stdout: ms inject socket failures = 5000 2021-02-20T13:08:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: fsid = f8fe48d9-618e-4952-943d-8a8e6ffe912c 2021-02-20T13:08:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.711 INFO:teuthology.orchestra.run.smithi183.stdout:[osd] 2021-02-20T13:08:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: osd journal size = 100 2021-02-20T13:08:20.712 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.712 INFO:teuthology.orchestra.run.smithi183.stdout: osd scrub load threshold = 5.0 2021-02-20T13:08:20.712 INFO:teuthology.orchestra.run.smithi183.stdout: osd scrub max interval = 600 2021-02-20T13:08:20.712 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.712 INFO:teuthology.orchestra.run.smithi183.stdout: osd recover clone overlap = true 2021-02-20T13:08:20.713 INFO:teuthology.orchestra.run.smithi183.stdout: osd recovery max chunk = 1048576 2021-02-20T13:08:20.713 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.713 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug shutdown = true 2021-02-20T13:08:20.713 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug op order = true 2021-02-20T13:08:20.713 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug verify stray on activate = true 2021-02-20T13:08:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: osd open classes on start = true 2021-02-20T13:08:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug pg log writeout = true 2021-02-20T13:08:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: osd deep scrub update digest min age = 30 2021-02-20T13:08:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: osd map max advance = 10 2021-02-20T13:08:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: journal zero on create = true 2021-02-20T13:08:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: filestore ondisk finisher threads = 3 2021-02-20T13:08:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: filestore apply finisher threads = 3 2021-02-20T13:08:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: bdev debug aio = true 2021-02-20T13:08:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug misdirected ops = true 2021-02-20T13:08:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore block size = 96636764160 2021-02-20T13:08:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore compression algorithm = lz4 2021-02-20T13:08:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore compression mode = aggressive 2021-02-20T13:08:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore fsck on mount = True 2021-02-20T13:08:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: debug bluefs = 20 2021-02-20T13:08:20.718 INFO:teuthology.orchestra.run.smithi183.stdout: debug bluestore = 20 2021-02-20T13:08:20.718 INFO:teuthology.orchestra.run.smithi183.stdout: debug filestore = 20 2021-02-20T13:08:20.718 INFO:teuthology.orchestra.run.smithi183.stdout: debug journal = 20 2021-02-20T13:08:20.718 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 20 2021-02-20T13:08:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: debug osd = 25 2021-02-20T13:08:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: debug rocksdb = 10 2021-02-20T13:08:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd backfillfull_ratio = 0.85 2021-02-20T13:08:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd full ratio = 0.9 2021-02-20T13:08:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd nearfull ratio = 0.8 2021-02-20T13:08:20.720 INFO:teuthology.orchestra.run.smithi183.stdout: osd failsafe full ratio = 0.95 2021-02-20T13:08:20.720 INFO:teuthology.orchestra.run.smithi183.stdout: osd objectstore = bluestore 2021-02-20T13:08:20.720 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.720 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr] 2021-02-20T13:08:20.720 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2021-02-20T13:08:20.721 INFO:teuthology.orchestra.run.smithi183.stdout: debug mgr = 20 2021-02-20T13:08:20.721 INFO:teuthology.orchestra.run.smithi183.stdout: debug mon = 20 2021-02-20T13:08:20.721 INFO:teuthology.orchestra.run.smithi183.stdout: debug auth = 20 2021-02-20T13:08:20.721 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min pgs per osd = 4 2021-02-20T13:08:20.721 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min bytes per osd = 10 2021-02-20T13:08:20.722 INFO:teuthology.orchestra.run.smithi183.stdout: mgr/telemetry/nag = false 2021-02-20T13:08:20.722 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.722 INFO:teuthology.orchestra.run.smithi183.stdout:[mon] 2021-02-20T13:08:20.722 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2021-02-20T13:08:20.722 INFO:teuthology.orchestra.run.smithi183.stdout: debug mon = 20 2021-02-20T13:08:20.723 INFO:teuthology.orchestra.run.smithi183.stdout: debug paxos = 20 2021-02-20T13:08:20.723 INFO:teuthology.orchestra.run.smithi183.stdout: debug auth = 20 2021-02-20T13:08:20.723 INFO:teuthology.orchestra.run.smithi183.stdout: mon data avail warn = 5 2021-02-20T13:08:20.723 INFO:teuthology.orchestra.run.smithi183.stdout: mon mgr mkfs grace = 240 2021-02-20T13:08:20.723 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min pgs per osd = 4 2021-02-20T13:08:20.724 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd reporter subtree level = osd 2021-02-20T13:08:20.724 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd prime pg temp = true 2021-02-20T13:08:20.724 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min bytes per osd = 10 2021-02-20T13:08:20.724 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:20.725 INFO:teuthology.orchestra.run.smithi183.stdout:[client] 2021-02-20T13:08:20.725 INFO:teuthology.orchestra.run.smithi183.stdout: rgw cache enabled = true 2021-02-20T13:08:20.725 INFO:teuthology.orchestra.run.smithi183.stdout: rgw enable ops log = true 2021-02-20T13:08:20.726 INFO:teuthology.orchestra.run.smithi183.stdout: rgw enable usage log = true 2021-02-20T13:08:20.726 INFO:teuthology.orchestra.run.smithi183.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-20T13:08:20.726 INFO:teuthology.orchestra.run.smithi183.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-20T13:08:20.726 INFO:teuthology.orchestra.run.smithi183.stdout: debug rbd = 20 2021-02-20T13:08:20.726 INFO:teuthology.orchestra.run.smithi183.stdout: debug rbd_mirror = 15 2021-02-20T13:08:20.727 INFO:teuthology.orchestra.run.smithi183.stdout: log to stderr = False 2021-02-20T13:08:20.727 INFO:teuthology.orchestra.run.smithi183.stdout: rbd default features = 125 2021-02-20T13:08:20.727 INFO:teuthology.orchestra.run.smithi183.stdout: rbd mirror image policy type = none 2021-02-20T13:08:20.727 INFO:teuthology.orchestra.run.smithi183.stdout:[mon.a] 2021-02-20T13:08:20.727 INFO:teuthology.orchestra.run.smithi183.stdout:[client.mirror.0] 2021-02-20T13:08:20.728 INFO:teuthology.orchestra.run.smithi183.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:08:20.728 INFO:teuthology.orchestra.run.smithi183.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:08:20.728 INFO:teuthology.orchestra.run.smithi183.stdout:[client.mirror.1] 2021-02-20T13:08:20.728 INFO:teuthology.orchestra.run.smithi183.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:08:20.728 INFO:teuthology.orchestra.run.smithi183.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:08:20.729 INFO:teuthology.orchestra.run.smithi183.stdout:[client.mirror.2] 2021-02-20T13:08:20.729 INFO:teuthology.orchestra.run.smithi183.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:08:20.729 INFO:teuthology.orchestra.run.smithi183.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:08:20.730 INFO:teuthology.orchestra.run.smithi183.stdout:[client.mirror.3] 2021-02-20T13:08:20.730 INFO:teuthology.orchestra.run.smithi183.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T13:08:20.730 INFO:teuthology.orchestra.run.smithi183.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T13:08:20.731 INFO:tasks.ceph:Creating admin key on cluster2.mon.a... 2021-02-20T13:08:20.731 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/cluster2.keyring 2021-02-20T13:08:20.787 INFO:tasks.ceph:Copying monmap to all nodes... 2021-02-20T13:08:20.788 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:20.788 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/cluster2.keyring of=/dev/stdout 2021-02-20T13:08:20.794 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:20.795 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/home/ubuntu/cephtest/cluster2.monmap of=/dev/stdout 2021-02-20T13:08:20.844 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi170.front.sepia.ceph.com 2021-02-20T13:08:20.844 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:08:20.845 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/cluster2.keyring 2021-02-20T13:08:20.845 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2021-02-20T13:08:20.875 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:08:20.876 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/home/ubuntu/cephtest/cluster2.monmap 2021-02-20T13:08:20.926 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi183.front.sepia.ceph.com 2021-02-20T13:08:20.927 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:20.927 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/cluster2.keyring 2021-02-20T13:08:20.928 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2021-02-20T13:08:20.953 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:20.954 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/home/ubuntu/cephtest/cluster2.monmap 2021-02-20T13:08:21.005 INFO:tasks.ceph:Setting up mon nodes... 2021-02-20T13:08:21.005 INFO:tasks.ceph:Setting up mgr nodes... 2021-02-20T13:08:21.006 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mgr/cluster2-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/cluster2-x/keyring 2021-02-20T13:08:21.048 INFO:teuthology.orchestra.run.smithi170.stdout:creating /var/lib/ceph/mgr/cluster2-x/keyring 2021-02-20T13:08:21.050 INFO:tasks.ceph:Setting up mds nodes... 2021-02-20T13:08:21.050 INFO:tasks.ceph_client:Setting up client nodes... 2021-02-20T13:08:21.051 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/cluster2.client.0.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.0.keyring 2021-02-20T13:08:21.124 INFO:teuthology.orchestra.run.smithi170.stdout:creating /etc/ceph/cluster2.client.0.keyring 2021-02-20T13:08:21.137 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror /etc/ceph/cluster2.client.mirror.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.keyring 2021-02-20T13:08:21.171 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/cluster2.client.mirror.keyring 2021-02-20T13:08:21.183 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.0 /etc/ceph/cluster2.client.mirror.0.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.0.keyring 2021-02-20T13:08:21.263 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/cluster2.client.mirror.0.keyring 2021-02-20T13:08:21.275 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.1 /etc/ceph/cluster2.client.mirror.1.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.1.keyring 2021-02-20T13:08:21.355 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/cluster2.client.mirror.1.keyring 2021-02-20T13:08:21.367 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.2 /etc/ceph/cluster2.client.mirror.2.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.2.keyring 2021-02-20T13:08:21.442 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/cluster2.client.mirror.2.keyring 2021-02-20T13:08:21.455 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.3 /etc/ceph/cluster2.client.mirror.3.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.3.keyring 2021-02-20T13:08:21.531 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/cluster2.client.mirror.3.keyring 2021-02-20T13:08:21.543 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.4 /etc/ceph/cluster2.client.mirror.4.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.4.keyring 2021-02-20T13:08:21.621 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/cluster2.client.mirror.4.keyring 2021-02-20T13:08:21.634 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.5 /etc/ceph/cluster2.client.mirror.5.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.5.keyring 2021-02-20T13:08:21.728 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/cluster2.client.mirror.5.keyring 2021-02-20T13:08:21.735 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.6 /etc/ceph/cluster2.client.mirror.6.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.6.keyring 2021-02-20T13:08:21.837 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/cluster2.client.mirror.6.keyring 2021-02-20T13:08:21.845 INFO:tasks.ceph:Running mkfs on osd nodes... 2021-02-20T13:08:21.846 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi170.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi183.front.sepia.ceph.com'): {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'}} 2021-02-20T13:08:21.846 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/cluster2-0 2021-02-20T13:08:21.905 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T13:08:21.905 INFO:tasks.ceph:role: cluster2.osd.0 2021-02-20T13:08:21.906 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi183.front.sepia.ceph.com 2021-02-20T13:08:21.907 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-02-20T13:08:22.437 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2021-02-20T13:08:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-20T13:08:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-02-20T13:08:22.438 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-02-20T13:08:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2021-02-20T13:08:22.438 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-20T13:08:22.439 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-02-20T13:08:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-20T13:08:22.439 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-20T13:08:22.440 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2021-02-20T13:08:22.441 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/cluster2-0 2021-02-20T13:08:22.526 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-0 2021-02-20T13:08:22.540 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: /sbin/restorecon: command not found 2021-02-20T13:08:22.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:08:22.542 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/cluster2-1 2021-02-20T13:08:22.598 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T13:08:22.598 INFO:tasks.ceph:role: cluster2.osd.1 2021-02-20T13:08:22.598 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi183.front.sepia.ceph.com 2021-02-20T13:08:22.599 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-02-20T13:08:23.101 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2021-02-20T13:08:23.101 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-20T13:08:23.102 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-02-20T13:08:23.102 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-02-20T13:08:23.102 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2021-02-20T13:08:23.102 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-20T13:08:23.103 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-02-20T13:08:23.103 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-20T13:08:23.103 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-20T13:08:23.104 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2021-02-20T13:08:23.105 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/cluster2-1 2021-02-20T13:08:23.158 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-1 2021-02-20T13:08:23.169 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: /sbin/restorecon: command not found 2021-02-20T13:08:23.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:08:23.170 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/cluster2-2 2021-02-20T13:08:23.223 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T13:08:23.224 INFO:tasks.ceph:role: cluster2.osd.2 2021-02-20T13:08:23.224 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi183.front.sepia.ceph.com 2021-02-20T13:08:23.225 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-02-20T13:08:23.714 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2021-02-20T13:08:23.715 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-20T13:08:23.715 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-02-20T13:08:23.715 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-02-20T13:08:23.716 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2021-02-20T13:08:23.716 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-20T13:08:23.716 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-02-20T13:08:23.716 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-20T13:08:23.717 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-20T13:08:23.718 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2021-02-20T13:08:23.719 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/cluster2-2 2021-02-20T13:08:23.770 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-2 2021-02-20T13:08:23.825 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: /sbin/restorecon: command not found 2021-02-20T13:08:23.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:08:23.829 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/cluster2.monmap 2021-02-20T13:08:23.920 INFO:teuthology.orchestra.run.smithi183.stderr:2021-02-20T13:08:23.917+0000 7f1c6ab43080 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-0/keyring: can't open /var/lib/ceph/osd/cluster2-0/keyring: (2) No such file or directory 2021-02-20T13:08:23.921 INFO:teuthology.orchestra.run.smithi183.stderr:2021-02-20T13:08:23.917+0000 7f1c6ab43080 -1 created new key in keyring /var/lib/ceph/osd/cluster2-0/keyring 2021-02-20T13:08:23.921 INFO:teuthology.orchestra.run.smithi183.stderr:2021-02-20T13:08:23.917+0000 7f1c6ab43080 -1 bluestore(/var/lib/ceph/osd/cluster2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-0/block: (2) No such file or directory 2021-02-20T13:08:23.921 INFO:teuthology.orchestra.run.smithi183.stderr:2021-02-20T13:08:23.917+0000 7f1c6ab43080 -1 bluestore(/var/lib/ceph/osd/cluster2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-0/block: (2) No such file or directory 2021-02-20T13:08:23.921 INFO:teuthology.orchestra.run.smithi183.stderr:2021-02-20T13:08:23.917+0000 7f1c6ab43080 -1 bluestore(/var/lib/ceph/osd/cluster2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-0/block: (2) No such file or directory 2021-02-20T13:08:23.928 INFO:teuthology.orchestra.run.smithi183.stderr:2021-02-20T13:08:23.925+0000 7f1c6ab43080 -1 bluestore(/var/lib/ceph/osd/cluster2-0) _read_fsid unparsable uuid 2021-02-20T13:08:26.871 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-0 2021-02-20T13:08:26.886 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/cluster2.monmap 2021-02-20T13:08:26.969 INFO:teuthology.orchestra.run.smithi183.stderr:2021-02-20T13:08:26.965+0000 7f5247ddb080 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-1/keyring: can't open /var/lib/ceph/osd/cluster2-1/keyring: (2) No such file or directory 2021-02-20T13:08:26.970 INFO:teuthology.orchestra.run.smithi183.stderr:2021-02-20T13:08:26.965+0000 7f5247ddb080 -1 created new key in keyring /var/lib/ceph/osd/cluster2-1/keyring 2021-02-20T13:08:26.970 INFO:teuthology.orchestra.run.smithi183.stderr:2021-02-20T13:08:26.965+0000 7f5247ddb080 -1 bluestore(/var/lib/ceph/osd/cluster2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-1/block: (2) No such file or directory 2021-02-20T13:08:26.970 INFO:teuthology.orchestra.run.smithi183.stderr:2021-02-20T13:08:26.965+0000 7f5247ddb080 -1 bluestore(/var/lib/ceph/osd/cluster2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-1/block: (2) No such file or directory 2021-02-20T13:08:26.970 INFO:teuthology.orchestra.run.smithi183.stderr:2021-02-20T13:08:26.965+0000 7f5247ddb080 -1 bluestore(/var/lib/ceph/osd/cluster2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-1/block: (2) No such file or directory 2021-02-20T13:08:26.971 INFO:teuthology.orchestra.run.smithi183.stderr:2021-02-20T13:08:26.969+0000 7f5247ddb080 -1 bluestore(/var/lib/ceph/osd/cluster2-1) _read_fsid unparsable uuid 2021-02-20T13:08:29.930 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-1 2021-02-20T13:08:29.989 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/cluster2.monmap 2021-02-20T13:08:30.068 INFO:teuthology.orchestra.run.smithi183.stderr:2021-02-20T13:08:30.065+0000 7f4c8e100080 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-2/keyring: can't open /var/lib/ceph/osd/cluster2-2/keyring: (2) No such file or directory 2021-02-20T13:08:30.068 INFO:teuthology.orchestra.run.smithi183.stderr:2021-02-20T13:08:30.065+0000 7f4c8e100080 -1 created new key in keyring /var/lib/ceph/osd/cluster2-2/keyring 2021-02-20T13:08:30.068 INFO:teuthology.orchestra.run.smithi183.stderr:2021-02-20T13:08:30.065+0000 7f4c8e100080 -1 bluestore(/var/lib/ceph/osd/cluster2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-2/block: (2) No such file or directory 2021-02-20T13:08:30.069 INFO:teuthology.orchestra.run.smithi183.stderr:2021-02-20T13:08:30.065+0000 7f4c8e100080 -1 bluestore(/var/lib/ceph/osd/cluster2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-2/block: (2) No such file or directory 2021-02-20T13:08:30.069 INFO:teuthology.orchestra.run.smithi183.stderr:2021-02-20T13:08:30.065+0000 7f4c8e100080 -1 bluestore(/var/lib/ceph/osd/cluster2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-2/block: (2) No such file or directory 2021-02-20T13:08:30.070 INFO:teuthology.orchestra.run.smithi183.stderr:2021-02-20T13:08:30.065+0000 7f4c8e100080 -1 bluestore(/var/lib/ceph/osd/cluster2-2) _read_fsid unparsable uuid 2021-02-20T13:08:33.014 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-2 2021-02-20T13:08:33.071 INFO:tasks.ceph:Reading keys from all nodes... 2021-02-20T13:08:33.072 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:08:33.073 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/mgr/cluster2-x/keyring of=/dev/stdout 2021-02-20T13:08:33.088 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:33.089 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/cluster2-0/keyring of=/dev/stdout 2021-02-20T13:08:33.133 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:33.134 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/cluster2-1/keyring of=/dev/stdout 2021-02-20T13:08:33.188 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:33.189 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/cluster2-2/keyring of=/dev/stdout 2021-02-20T13:08:33.244 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2021-02-20T13:08:33.245 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/etc/ceph/cluster2.client.0.keyring of=/dev/stdout 2021-02-20T13:08:33.265 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:33.266 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/cluster2.client.mirror.keyring of=/dev/stdout 2021-02-20T13:08:33.291 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:33.292 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/cluster2.client.mirror.0.keyring of=/dev/stdout 2021-02-20T13:08:33.339 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:33.340 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/cluster2.client.mirror.1.keyring of=/dev/stdout 2021-02-20T13:08:33.388 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:33.388 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/cluster2.client.mirror.2.keyring of=/dev/stdout 2021-02-20T13:08:33.437 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:33.438 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/cluster2.client.mirror.3.keyring of=/dev/stdout 2021-02-20T13:08:33.484 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:33.485 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/cluster2.client.mirror.4.keyring of=/dev/stdout 2021-02-20T13:08:33.532 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:33.533 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/cluster2.client.mirror.5.keyring of=/dev/stdout 2021-02-20T13:08:33.579 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:33.580 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/cluster2.client.mirror.6.keyring of=/dev/stdout 2021-02-20T13:08:33.629 INFO:tasks.ceph:Adding keys to all mons... 2021-02-20T13:08:33.630 DEBUG:teuthology.orchestra.run.smithi183:> sudo tee -a /etc/ceph/cluster2.keyring 2021-02-20T13:08:33.683 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr.x] 2021-02-20T13:08:33.684 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDFCTFgDYEEAxAAkOnNstaoyhu2Si79p+Ycxw== 2021-02-20T13:08:33.684 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.0] 2021-02-20T13:08:33.685 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDHCTFgDsUBNxAAxcRDBXzuQqDpCXgdQVoP4w== 2021-02-20T13:08:33.686 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.1] 2021-02-20T13:08:33.686 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDKCTFg7M3xORAAIlgONrMA9s5BcjjZFH1qGg== 2021-02-20T13:08:33.686 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.2] 2021-02-20T13:08:33.687 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDOCTFgtt00BBAA3yEtt2WDDSFGulqMUHFtlA== 2021-02-20T13:08:33.687 INFO:teuthology.orchestra.run.smithi183.stdout:[client.0] 2021-02-20T13:08:33.688 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDFCTFgcaCOBxAAHZk+T/tZPvNNATj8jPgXZA== 2021-02-20T13:08:33.688 INFO:teuthology.orchestra.run.smithi183.stdout:[client.mirror] 2021-02-20T13:08:33.689 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDFCTFgbLVgChAA4HU6HBUTp8nmsqS74tM2og== 2021-02-20T13:08:33.689 INFO:teuthology.orchestra.run.smithi183.stdout:[client.mirror.0] 2021-02-20T13:08:33.689 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDFCTFgknHYDxAAvK38hWM23z37DKO8W36dhg== 2021-02-20T13:08:33.690 INFO:teuthology.orchestra.run.smithi183.stdout:[client.mirror.1] 2021-02-20T13:08:33.690 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDFCTFgdopVFRAAijwQl8alTNvn2Ue3yyWIKA== 2021-02-20T13:08:33.691 INFO:teuthology.orchestra.run.smithi183.stdout:[client.mirror.2] 2021-02-20T13:08:33.691 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDFCTFgP8qMGhAAALKo8MPOb9/VOrxaCfVtJg== 2021-02-20T13:08:33.692 INFO:teuthology.orchestra.run.smithi183.stdout:[client.mirror.3] 2021-02-20T13:08:33.692 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDFCTFg54LXHxAAvTLudJdXvwTwaWowiCmBYQ== 2021-02-20T13:08:33.693 INFO:teuthology.orchestra.run.smithi183.stdout:[client.mirror.4] 2021-02-20T13:08:33.694 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDFCTFg44MzJRAAjC7Pm1DG6RPocDeDQKVHfA== 2021-02-20T13:08:33.694 INFO:teuthology.orchestra.run.smithi183.stdout:[client.mirror.5] 2021-02-20T13:08:33.694 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDFCTFgj8s9KxAAD74FZk0bYGT5R3kB63lb5w== 2021-02-20T13:08:33.695 INFO:teuthology.orchestra.run.smithi183.stdout:[client.mirror.6] 2021-02-20T13:08:33.695 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDFCTFg8R63MRAApiTJwEywG2ZB2SprjbWP5A== 2021-02-20T13:08:33.698 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-02-20T13:08:33.784 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-20T13:08:33.876 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-20T13:08:33.963 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-20T13:08:34.048 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T13:08:34.134 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T13:08:34.222 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T13:08:34.265 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T13:08:34.351 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T13:08:34.439 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T13:08:34.481 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.4 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T13:08:34.567 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.5 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T13:08:34.608 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.6 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T13:08:34.697 INFO:tasks.ceph:Running mkfs on mon nodes... 2021-02-20T13:08:34.698 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mon/cluster2-a 2021-02-20T13:08:34.710 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster cluster2 --mkfs -i a --monmap /home/ubuntu/cephtest/cluster2.monmap --keyring /etc/ceph/cluster2.keyring 2021-02-20T13:08:34.808 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/mon/cluster2-a 2021-02-20T13:08:34.822 DEBUG:teuthology.orchestra.run.smithi183:> rm -- /home/ubuntu/cephtest/cluster2.monmap 2021-02-20T13:08:34.871 INFO:tasks.ceph:Starting mon daemons in cluster cluster2... 2021-02-20T13:08:34.872 INFO:tasks.ceph.cluster2.mon.a:Restarting daemon 2021-02-20T13:08:34.872 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster cluster2 -i a 2021-02-20T13:08:34.915 INFO:tasks.ceph.cluster2.mon.a:Started 2021-02-20T13:08:34.916 INFO:tasks.ceph:Starting mgr daemons in cluster cluster2... 2021-02-20T13:08:34.916 INFO:tasks.ceph.cluster2.mgr.x:Restarting daemon 2021-02-20T13:08:34.917 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster cluster2 -i x 2021-02-20T13:08:34.920 INFO:tasks.ceph.cluster2.mgr.x:Started 2021-02-20T13:08:34.920 INFO:tasks.ceph:Setting crush tunables to default 2021-02-20T13:08:34.921 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster cluster2 osd crush tunables default 2021-02-20T13:08:35.234 INFO:teuthology.orchestra.run.smithi183.stderr:adjusted tunables profile to default 2021-02-20T13:08:35.248 INFO:tasks.ceph:Starting osd daemons in cluster cluster2... 2021-02-20T13:08:35.248 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:35.248 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/cluster2-0/fsid of=/dev/stdout 2021-02-20T13:08:35.263 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:35.263 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/cluster2-1/fsid of=/dev/stdout 2021-02-20T13:08:35.322 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:08:35.323 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/cluster2-2/fsid of=/dev/stdout 2021-02-20T13:08:35.385 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster cluster2 osd new 4b7c9c8c-3434-48f1-9e7b-2f7a040fd69b 0 2021-02-20T13:08:35.660 INFO:teuthology.orchestra.run.smithi183.stdout:0 2021-02-20T13:08:35.674 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster cluster2 osd new fcc91e56-df23-4dcb-b5f4-c074fd061b0a 1 2021-02-20T13:08:35.914 INFO:teuthology.orchestra.run.smithi183.stdout:1 2021-02-20T13:08:35.927 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster cluster2 osd new 11818e68-aded-4372-880b-13a1c554c07c 2 2021-02-20T13:08:36.158 INFO:teuthology.orchestra.run.smithi183.stdout:2 2021-02-20T13:08:36.173 INFO:tasks.ceph.cluster2.osd.0:Restarting daemon 2021-02-20T13:08:36.173 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 0 2021-02-20T13:08:36.175 INFO:tasks.ceph.cluster2.osd.0:Started 2021-02-20T13:08:36.176 INFO:tasks.ceph.cluster2.osd.1:Restarting daemon 2021-02-20T13:08:36.176 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 1 2021-02-20T13:08:36.178 INFO:tasks.ceph.cluster2.osd.1:Started 2021-02-20T13:08:36.179 INFO:tasks.ceph.cluster2.osd.2:Restarting daemon 2021-02-20T13:08:36.179 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 2 2021-02-20T13:08:36.182 INFO:tasks.ceph.cluster2.osd.2:Started 2021-02-20T13:08:36.182 INFO:tasks.ceph:Waiting for OSDs to come up 2021-02-20T13:08:36.486 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster2 osd dump --format=json 2021-02-20T13:08:36.691 INFO:teuthology.misc.health.smithi183.stdout: 2021-02-20T13:08:36.691 INFO:teuthology.misc.health.smithi183.stdout:{"epoch":5,"fsid":"f8fe48d9-618e-4952-943d-8a8e6ffe912c","created":"2021-02-20T13:08:35.044820+0000","modified":"2021-02-20T13:08:36.158261+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"4b7c9c8c-3434-48f1-9e7b-2f7a040fd69b","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fcc91e56-df23-4dcb-b5f4-c074fd061b0a","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"11818e68-aded-4372-880b-13a1c554c07c","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T13:08:36.703 DEBUG:teuthology.misc:0 of 3 OSDs are up 2021-02-20T13:08:36.788 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T13:08:36.785+0000 7fcbe830f080 -1 Falling back to public interface 2021-02-20T13:08:36.812 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T13:08:36.809+0000 7f4a327cc080 -1 Falling back to public interface 2021-02-20T13:08:36.823 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T13:08:36.821+0000 7f03e3992080 -1 Falling back to public interface 2021-02-20T13:08:40.487 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T13:08:40.485+0000 7fcbe830f080 -1 osd.2 0 log_to_monitors {default=true} 2021-02-20T13:08:40.585 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T13:08:40.581+0000 7f4a327cc080 -1 osd.0 0 log_to_monitors {default=true} 2021-02-20T13:08:40.624 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T13:08:40.621+0000 7f03e3992080 -1 osd.1 0 log_to_monitors {default=true} 2021-02-20T13:08:42.061 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T13:08:42.057+0000 7fcbdf7d8700 -1 osd.2 0 waiting for initial osdmap 2021-02-20T13:08:42.062 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T13:08:42.061+0000 7f4a29c95700 -1 osd.0 0 waiting for initial osdmap 2021-02-20T13:08:42.064 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T13:08:42.061+0000 7f03dae5b700 -1 osd.1 0 waiting for initial osdmap 2021-02-20T13:08:43.007 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster2 osd dump --format=json 2021-02-20T13:08:43.301 INFO:teuthology.misc.health.smithi183.stdout: 2021-02-20T13:08:43.302 INFO:teuthology.misc.health.smithi183.stdout:{"epoch":8,"fsid":"f8fe48d9-618e-4952-943d-8a8e6ffe912c","created":"2021-02-20T13:08:35.044820+0000","modified":"2021-02-20T13:08:43.056530+0000","last_up_change":"2021-02-20T13:08:43.056530+0000","last_in_change":"2021-02-20T13:08:43.056530+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"4b7c9c8c-3434-48f1-9e7b-2f7a040fd69b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15828},{"type":"v1","addr":"172.21.15.183:6809","nonce":15828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15828},{"type":"v1","addr":"172.21.15.183:6811","nonce":15828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15828},{"type":"v1","addr":"172.21.15.183:6815","nonce":15828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15828},{"type":"v1","addr":"172.21.15.183:6813","nonce":15828}]},"public_addr":"172.21.15.183:6809/15828","cluster_addr":"172.21.15.183:6811/15828","heartbeat_back_addr":"172.21.15.183:6815/15828","heartbeat_front_addr":"172.21.15.183:6813/15828","state":["exists","up"]},{"osd":1,"uuid":"fcc91e56-df23-4dcb-b5f4-c074fd061b0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":15827},{"type":"v1","addr":"172.21.15.183:6817","nonce":15827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":15827},{"type":"v1","addr":"172.21.15.183:6819","nonce":15827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":15827},{"type":"v1","addr":"172.21.15.183:6823","nonce":15827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":15827},{"type":"v1","addr":"172.21.15.183:6821","nonce":15827}]},"public_addr":"172.21.15.183:6817/15827","cluster_addr":"172.21.15.183:6819/15827","heartbeat_back_addr":"172.21.15.183:6823/15827","heartbeat_front_addr":"172.21.15.183:6821/15827","state":["exists","up"]},{"osd":2,"uuid":"11818e68-aded-4372-880b-13a1c554c07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":15829},{"type":"v1","addr":"172.21.15.183:6801","nonce":15829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":15829},{"type":"v1","addr":"172.21.15.183:6803","nonce":15829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":15829},{"type":"v1","addr":"172.21.15.183:6807","nonce":15829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":15829},{"type":"v1","addr":"172.21.15.183:6805","nonce":15829}]},"public_addr":"172.21.15.183:6801/15829","cluster_addr":"172.21.15.183:6803/15829","heartbeat_back_addr":"172.21.15.183:6807/15829","heartbeat_front_addr":"172.21.15.183:6805/15829","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T13:08:43.315 DEBUG:teuthology.misc:3 of 3 OSDs are up 2021-02-20T13:08:43.317 INFO:tasks.ceph:Creating RBD pool 2021-02-20T13:08:43.317 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster cluster2 osd pool create rbd 8 2021-02-20T13:08:44.084 INFO:teuthology.orchestra.run.smithi183.stderr:pool 'rbd' created 2021-02-20T13:08:44.098 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster cluster2 osd pool application enable rbd rbd --yes-i-really-mean-it 2021-02-20T13:08:45.092 INFO:teuthology.orchestra.run.smithi183.stderr:enabled application 'rbd' on pool 'rbd' 2021-02-20T13:08:45.107 INFO:tasks.ceph:Starting mds daemons in cluster cluster2... 2021-02-20T13:08:45.107 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2021-02-20T13:08:45.110 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2021-02-20T13:08:45.413 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:45.414 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":10,"fsid":"f8fe48d9-618e-4952-943d-8a8e6ffe912c","created":"2021-02-20T13:08:35.044820+0000","modified":"2021-02-20T13:08:45.075270+0000","last_up_change":"2021-02-20T13:08:43.056530+0000","last_in_change":"2021-02-20T13:08:43.056530+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-02-20T13:08:43.129746+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-02-20T13:08:43.690985+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"4b7c9c8c-3434-48f1-9e7b-2f7a040fd69b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15828},{"type":"v1","addr":"172.21.15.183:6809","nonce":15828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15828},{"type":"v1","addr":"172.21.15.183:6811","nonce":15828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15828},{"type":"v1","addr":"172.21.15.183:6815","nonce":15828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15828},{"type":"v1","addr":"172.21.15.183:6813","nonce":15828}]},"public_addr":"172.21.15.183:6809/15828","cluster_addr":"172.21.15.183:6811/15828","heartbeat_back_addr":"172.21.15.183:6815/15828","heartbeat_front_addr":"172.21.15.183:6813/15828","state":["exists","up"]},{"osd":1,"uuid":"fcc91e56-df23-4dcb-b5f4-c074fd061b0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":15827},{"type":"v1","addr":"172.21.15.183:6817","nonce":15827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":15827},{"type":"v1","addr":"172.21.15.183:6819","nonce":15827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":15827},{"type":"v1","addr":"172.21.15.183:6823","nonce":15827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":15827},{"type":"v1","addr":"172.21.15.183:6821","nonce":15827}]},"public_addr":"172.21.15.183:6817/15827","cluster_addr":"172.21.15.183:6819/15827","heartbeat_back_addr":"172.21.15.183:6823/15827","heartbeat_front_addr":"172.21.15.183:6821/15827","state":["exists","up"]},{"osd":2,"uuid":"11818e68-aded-4372-880b-13a1c554c07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":15829},{"type":"v1","addr":"172.21.15.183:6801","nonce":15829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":15829},{"type":"v1","addr":"172.21.15.183:6803","nonce":15829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":15829},{"type":"v1","addr":"172.21.15.183:6807","nonce":15829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":15829},{"type":"v1","addr":"172.21.15.183:6805","nonce":15829}]},"public_addr":"172.21.15.183:6801/15829","cluster_addr":"172.21.15.183:6803/15829","heartbeat_back_addr":"172.21.15.183:6807/15829","heartbeat_front_addr":"172.21.15.183:6805/15829","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T13:08:41.545383+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T13:08:41.617436+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T13:08:41.528060+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T13:08:45.428 INFO:tasks.ceph.ceph_manager.cluster2:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-02-20T13:08:43.129746+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-02-20T13:08:43.690985+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-20T13:08:45.429 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd pool get device_health_metrics pg_num 2021-02-20T13:08:45.735 INFO:teuthology.orchestra.run.smithi183.stdout:pg_num: 1 2021-02-20T13:08:45.749 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd pool get rbd pg_num 2021-02-20T13:08:46.052 INFO:teuthology.orchestra.run.smithi183.stdout:pg_num: 8 2021-02-20T13:08:46.066 INFO:tasks.ceph:Waiting until cluster2 daemons up and pgs clean... 2021-02-20T13:08:46.067 INFO:tasks.ceph.ceph_manager.cluster2:waiting for mgr available 2021-02-20T13:08:46.067 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 mgr dump --format=json 2021-02-20T13:08:46.409 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:46.423 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":4,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":18901},{"type":"v1","addr":"172.21.15.170:6827","nonce":18901}]},"active_addr":"172.21.15.170:6827/18901","active_change":"2021-02-20T13:08:38.985150+0000","active_mgr_features":4540138303579357183,"available":true,"standbys":[],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"haproxy","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":3553155504}]},{"addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":3798099672}]}]}} 2021-02-20T13:08:46.425 INFO:tasks.ceph.ceph_manager.cluster2:mgr available! 2021-02-20T13:08:46.426 INFO:tasks.ceph.ceph_manager.cluster2:waiting for all up 2021-02-20T13:08:46.426 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2021-02-20T13:08:46.721 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:46.721 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":11,"fsid":"f8fe48d9-618e-4952-943d-8a8e6ffe912c","created":"2021-02-20T13:08:35.044820+0000","modified":"2021-02-20T13:08:46.093375+0000","last_up_change":"2021-02-20T13:08:43.056530+0000","last_in_change":"2021-02-20T13:08:43.056530+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-02-20T13:08:43.129746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-02-20T13:08:43.690985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"4b7c9c8c-3434-48f1-9e7b-2f7a040fd69b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15828},{"type":"v1","addr":"172.21.15.183:6809","nonce":15828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15828},{"type":"v1","addr":"172.21.15.183:6811","nonce":15828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15828},{"type":"v1","addr":"172.21.15.183:6815","nonce":15828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15828},{"type":"v1","addr":"172.21.15.183:6813","nonce":15828}]},"public_addr":"172.21.15.183:6809/15828","cluster_addr":"172.21.15.183:6811/15828","heartbeat_back_addr":"172.21.15.183:6815/15828","heartbeat_front_addr":"172.21.15.183:6813/15828","state":["exists","up"]},{"osd":1,"uuid":"fcc91e56-df23-4dcb-b5f4-c074fd061b0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":15827},{"type":"v1","addr":"172.21.15.183:6817","nonce":15827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":15827},{"type":"v1","addr":"172.21.15.183:6819","nonce":15827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":15827},{"type":"v1","addr":"172.21.15.183:6823","nonce":15827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":15827},{"type":"v1","addr":"172.21.15.183:6821","nonce":15827}]},"public_addr":"172.21.15.183:6817/15827","cluster_addr":"172.21.15.183:6819/15827","heartbeat_back_addr":"172.21.15.183:6823/15827","heartbeat_front_addr":"172.21.15.183:6821/15827","state":["exists","up"]},{"osd":2,"uuid":"11818e68-aded-4372-880b-13a1c554c07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":15829},{"type":"v1","addr":"172.21.15.183:6801","nonce":15829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":15829},{"type":"v1","addr":"172.21.15.183:6803","nonce":15829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":15829},{"type":"v1","addr":"172.21.15.183:6807","nonce":15829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":15829},{"type":"v1","addr":"172.21.15.183:6805","nonce":15829}]},"public_addr":"172.21.15.183:6801/15829","cluster_addr":"172.21.15.183:6803/15829","heartbeat_back_addr":"172.21.15.183:6807/15829","heartbeat_front_addr":"172.21.15.183:6805/15829","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T13:08:41.545383+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T13:08:41.617436+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T13:08:41.528060+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T13:08:46.736 INFO:tasks.ceph.ceph_manager.cluster2:all up! 2021-02-20T13:08:46.737 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2021-02-20T13:08:47.038 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:47.039 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":11,"fsid":"f8fe48d9-618e-4952-943d-8a8e6ffe912c","created":"2021-02-20T13:08:35.044820+0000","modified":"2021-02-20T13:08:46.093375+0000","last_up_change":"2021-02-20T13:08:43.056530+0000","last_in_change":"2021-02-20T13:08:43.056530+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-02-20T13:08:43.129746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-02-20T13:08:43.690985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"4b7c9c8c-3434-48f1-9e7b-2f7a040fd69b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15828},{"type":"v1","addr":"172.21.15.183:6809","nonce":15828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15828},{"type":"v1","addr":"172.21.15.183:6811","nonce":15828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15828},{"type":"v1","addr":"172.21.15.183:6815","nonce":15828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15828},{"type":"v1","addr":"172.21.15.183:6813","nonce":15828}]},"public_addr":"172.21.15.183:6809/15828","cluster_addr":"172.21.15.183:6811/15828","heartbeat_back_addr":"172.21.15.183:6815/15828","heartbeat_front_addr":"172.21.15.183:6813/15828","state":["exists","up"]},{"osd":1,"uuid":"fcc91e56-df23-4dcb-b5f4-c074fd061b0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":15827},{"type":"v1","addr":"172.21.15.183:6817","nonce":15827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":15827},{"type":"v1","addr":"172.21.15.183:6819","nonce":15827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":15827},{"type":"v1","addr":"172.21.15.183:6823","nonce":15827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":15827},{"type":"v1","addr":"172.21.15.183:6821","nonce":15827}]},"public_addr":"172.21.15.183:6817/15827","cluster_addr":"172.21.15.183:6819/15827","heartbeat_back_addr":"172.21.15.183:6823/15827","heartbeat_front_addr":"172.21.15.183:6821/15827","state":["exists","up"]},{"osd":2,"uuid":"11818e68-aded-4372-880b-13a1c554c07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":15829},{"type":"v1","addr":"172.21.15.183:6801","nonce":15829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":15829},{"type":"v1","addr":"172.21.15.183:6803","nonce":15829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":15829},{"type":"v1","addr":"172.21.15.183:6807","nonce":15829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":15829},{"type":"v1","addr":"172.21.15.183:6805","nonce":15829}]},"public_addr":"172.21.15.183:6801/15829","cluster_addr":"172.21.15.183:6803/15829","heartbeat_back_addr":"172.21.15.183:6807/15829","heartbeat_front_addr":"172.21.15.183:6805/15829","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T13:08:41.545383+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T13:08:41.617436+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T13:08:41.528060+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T13:08:47.053 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.0 flush_pg_stats 2021-02-20T13:08:47.218 INFO:teuthology.orchestra.run.smithi183.stdout:34359738370 2021-02-20T13:08:47.219 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.1 flush_pg_stats 2021-02-20T13:08:47.391 INFO:teuthology.orchestra.run.smithi183.stdout:34359738370 2021-02-20T13:08:47.391 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.2 flush_pg_stats 2021-02-20T13:08:47.564 INFO:teuthology.orchestra.run.smithi183.stdout:34359738370 2021-02-20T13:08:47.565 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.0 2021-02-20T13:08:47.864 INFO:teuthology.orchestra.run.smithi183.stdout:34359738369 2021-02-20T13:08:47.878 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738370 got 34359738369 for osd.0 2021-02-20T13:08:48.880 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.0 2021-02-20T13:08:49.185 INFO:teuthology.orchestra.run.smithi183.stdout:34359738371 2021-02-20T13:08:49.200 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738370 got 34359738371 for osd.0 2021-02-20T13:08:49.200 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.1 2021-02-20T13:08:49.494 INFO:teuthology.orchestra.run.smithi183.stdout:34359738370 2021-02-20T13:08:49.508 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738370 got 34359738370 for osd.1 2021-02-20T13:08:49.509 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.2 2021-02-20T13:08:49.808 INFO:teuthology.orchestra.run.smithi183.stdout:34359738371 2021-02-20T13:08:49.822 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738370 got 34359738371 for osd.2 2021-02-20T13:08:49.823 INFO:tasks.ceph.ceph_manager.cluster2:waiting for clean 2021-02-20T13:08:49.823 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2021-02-20T13:08:50.117 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:50.119 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2021-02-20T13:08:50.131 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2021-02-20T13:08:48.993889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":14640,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":283100880,"statfs":{"total":289910292480,"available":289895301120,"internally_reserved":0,"allocated":286720,"data_stored":76172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:45.100633+0000","last_change":"2021-02-20T13:08:45.099283+0000","last_active":"2021-02-20T13:08:45.100633+0000","last_peered":"2021-02-20T13:08:45.100633+0000","last_clean":"2021-02-20T13:08:45.100633+0000","last_became_active":"2021-02-20T13:08:45.098026+0000","last_became_peered":"2021-02-20T13:08:45.098026+0000","last_unstale":"2021-02-20T13:08:45.100633+0000","last_undegraded":"2021-02-20T13:08:45.100633+0000","last_fullsized":"2021-02-20T13:08:45.100633+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_clean_scrub_stamp":"2021-02-20T13:08:44.066399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:45.100588+0000","last_change":"2021-02-20T13:08:45.099174+0000","last_active":"2021-02-20T13:08:45.100588+0000","last_peered":"2021-02-20T13:08:45.100588+0000","last_clean":"2021-02-20T13:08:45.100588+0000","last_became_active":"2021-02-20T13:08:45.098188+0000","last_became_peered":"2021-02-20T13:08:45.098188+0000","last_unstale":"2021-02-20T13:08:45.100588+0000","last_undegraded":"2021-02-20T13:08:45.100588+0000","last_fullsized":"2021-02-20T13:08:45.100588+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_clean_scrub_stamp":"2021-02-20T13:08:44.066399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:45.101104+0000","last_change":"2021-02-20T13:08:45.100291+0000","last_active":"2021-02-20T13:08:45.101104+0000","last_peered":"2021-02-20T13:08:45.101104+0000","last_clean":"2021-02-20T13:08:45.101104+0000","last_became_active":"2021-02-20T13:08:45.099954+0000","last_became_peered":"2021-02-20T13:08:45.099954+0000","last_unstale":"2021-02-20T13:08:45.101104+0000","last_undegraded":"2021-02-20T13:08:45.101104+0000","last_fullsized":"2021-02-20T13:08:45.101104+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_clean_scrub_stamp":"2021-02-20T13:08:44.066399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:45.096252+0000","last_change":"2021-02-20T13:08:45.096252+0000","last_active":"2021-02-20T13:08:45.096252+0000","last_peered":"2021-02-20T13:08:45.096252+0000","last_clean":"2021-02-20T13:08:45.096252+0000","last_became_active":"2021-02-20T13:08:45.095843+0000","last_became_peered":"2021-02-20T13:08:45.095843+0000","last_unstale":"2021-02-20T13:08:45.096252+0000","last_undegraded":"2021-02-20T13:08:45.096252+0000","last_fullsized":"2021-02-20T13:08:45.096252+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_clean_scrub_stamp":"2021-02-20T13:08:44.066399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:45.099951+0000","last_change":"2021-02-20T13:08:45.097567+0000","last_active":"2021-02-20T13:08:45.099951+0000","last_peered":"2021-02-20T13:08:45.099951+0000","last_clean":"2021-02-20T13:08:45.099951+0000","last_became_active":"2021-02-20T13:08:45.097210+0000","last_became_peered":"2021-02-20T13:08:45.097210+0000","last_unstale":"2021-02-20T13:08:45.099951+0000","last_undegraded":"2021-02-20T13:08:45.099951+0000","last_fullsized":"2021-02-20T13:08:45.099951+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_clean_scrub_stamp":"2021-02-20T13:08:44.066399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:45.100574+0000","last_change":"2021-02-20T13:08:45.098918+0000","last_active":"2021-02-20T13:08:45.100574+0000","last_peered":"2021-02-20T13:08:45.100574+0000","last_clean":"2021-02-20T13:08:45.100574+0000","last_became_active":"2021-02-20T13:08:45.097971+0000","last_became_peered":"2021-02-20T13:08:45.097971+0000","last_unstale":"2021-02-20T13:08:45.100574+0000","last_undegraded":"2021-02-20T13:08:45.100574+0000","last_fullsized":"2021-02-20T13:08:45.100574+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_clean_scrub_stamp":"2021-02-20T13:08:44.066399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:45.096289+0000","last_change":"2021-02-20T13:08:45.096289+0000","last_active":"2021-02-20T13:08:45.096289+0000","last_peered":"2021-02-20T13:08:45.096289+0000","last_clean":"2021-02-20T13:08:45.096289+0000","last_became_active":"2021-02-20T13:08:45.096004+0000","last_became_peered":"2021-02-20T13:08:45.096004+0000","last_unstale":"2021-02-20T13:08:45.096289+0000","last_undegraded":"2021-02-20T13:08:45.096289+0000","last_fullsized":"2021-02-20T13:08:45.096289+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_clean_scrub_stamp":"2021-02-20T13:08:44.066399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:45.095912+0000","last_change":"2021-02-20T13:08:45.094582+0000","last_active":"2021-02-20T13:08:45.095912+0000","last_peered":"2021-02-20T13:08:45.095912+0000","last_clean":"2021-02-20T13:08:45.095912+0000","last_became_active":"2021-02-20T13:08:45.094304+0000","last_became_peered":"2021-02-20T13:08:45.094304+0000","last_unstale":"2021-02-20T13:08:45.095912+0000","last_undegraded":"2021-02-20T13:08:45.095912+0000","last_fullsized":"2021-02-20T13:08:45.095912+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_clean_scrub_stamp":"2021-02-20T13:08:44.066399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:45.100615+0000","last_change":"2021-02-20T13:08:45.099198+0000","last_active":"2021-02-20T13:08:45.100615+0000","last_peered":"2021-02-20T13:08:45.100615+0000","last_clean":"2021-02-20T13:08:45.100615+0000","last_became_active":"2021-02-20T13:08:45.098155+0000","last_became_peered":"2021-02-20T13:08:45.098155+0000","last_unstale":"2021-02-20T13:08:45.100615+0000","last_undegraded":"2021-02-20T13:08:45.100615+0000","last_fullsized":"2021-02-20T13:08:45.100615+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_clean_scrub_stamp":"2021-02-20T13:08:44.066399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4832,"kb_used_data":88,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94367008,"statfs":{"total":96636764160,"available":96631816192,"internally_reserved":0,"allocated":90112,"data_stored":22804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4159999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0449999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]}]},{"osd":1,"up_from":8,"seq":34359738370,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]}]}]}} 2021-02-20T13:08:50.133 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2021-02-20T13:08:50.422 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:50.425 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2021-02-20T13:08:50.436 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2021-02-20T13:08:48.993889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":14640,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":283100880,"statfs":{"total":289910292480,"available":289895301120,"internally_reserved":0,"allocated":286720,"data_stored":76172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:45.100633+0000","last_change":"2021-02-20T13:08:45.099283+0000","last_active":"2021-02-20T13:08:45.100633+0000","last_peered":"2021-02-20T13:08:45.100633+0000","last_clean":"2021-02-20T13:08:45.100633+0000","last_became_active":"2021-02-20T13:08:45.098026+0000","last_became_peered":"2021-02-20T13:08:45.098026+0000","last_unstale":"2021-02-20T13:08:45.100633+0000","last_undegraded":"2021-02-20T13:08:45.100633+0000","last_fullsized":"2021-02-20T13:08:45.100633+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_clean_scrub_stamp":"2021-02-20T13:08:44.066399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:45.100588+0000","last_change":"2021-02-20T13:08:45.099174+0000","last_active":"2021-02-20T13:08:45.100588+0000","last_peered":"2021-02-20T13:08:45.100588+0000","last_clean":"2021-02-20T13:08:45.100588+0000","last_became_active":"2021-02-20T13:08:45.098188+0000","last_became_peered":"2021-02-20T13:08:45.098188+0000","last_unstale":"2021-02-20T13:08:45.100588+0000","last_undegraded":"2021-02-20T13:08:45.100588+0000","last_fullsized":"2021-02-20T13:08:45.100588+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_clean_scrub_stamp":"2021-02-20T13:08:44.066399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:45.101104+0000","last_change":"2021-02-20T13:08:45.100291+0000","last_active":"2021-02-20T13:08:45.101104+0000","last_peered":"2021-02-20T13:08:45.101104+0000","last_clean":"2021-02-20T13:08:45.101104+0000","last_became_active":"2021-02-20T13:08:45.099954+0000","last_became_peered":"2021-02-20T13:08:45.099954+0000","last_unstale":"2021-02-20T13:08:45.101104+0000","last_undegraded":"2021-02-20T13:08:45.101104+0000","last_fullsized":"2021-02-20T13:08:45.101104+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_clean_scrub_stamp":"2021-02-20T13:08:44.066399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:45.096252+0000","last_change":"2021-02-20T13:08:45.096252+0000","last_active":"2021-02-20T13:08:45.096252+0000","last_peered":"2021-02-20T13:08:45.096252+0000","last_clean":"2021-02-20T13:08:45.096252+0000","last_became_active":"2021-02-20T13:08:45.095843+0000","last_became_peered":"2021-02-20T13:08:45.095843+0000","last_unstale":"2021-02-20T13:08:45.096252+0000","last_undegraded":"2021-02-20T13:08:45.096252+0000","last_fullsized":"2021-02-20T13:08:45.096252+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_clean_scrub_stamp":"2021-02-20T13:08:44.066399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:45.099951+0000","last_change":"2021-02-20T13:08:45.097567+0000","last_active":"2021-02-20T13:08:45.099951+0000","last_peered":"2021-02-20T13:08:45.099951+0000","last_clean":"2021-02-20T13:08:45.099951+0000","last_became_active":"2021-02-20T13:08:45.097210+0000","last_became_peered":"2021-02-20T13:08:45.097210+0000","last_unstale":"2021-02-20T13:08:45.099951+0000","last_undegraded":"2021-02-20T13:08:45.099951+0000","last_fullsized":"2021-02-20T13:08:45.099951+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_clean_scrub_stamp":"2021-02-20T13:08:44.066399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:45.100574+0000","last_change":"2021-02-20T13:08:45.098918+0000","last_active":"2021-02-20T13:08:45.100574+0000","last_peered":"2021-02-20T13:08:45.100574+0000","last_clean":"2021-02-20T13:08:45.100574+0000","last_became_active":"2021-02-20T13:08:45.097971+0000","last_became_peered":"2021-02-20T13:08:45.097971+0000","last_unstale":"2021-02-20T13:08:45.100574+0000","last_undegraded":"2021-02-20T13:08:45.100574+0000","last_fullsized":"2021-02-20T13:08:45.100574+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_clean_scrub_stamp":"2021-02-20T13:08:44.066399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:45.096289+0000","last_change":"2021-02-20T13:08:45.096289+0000","last_active":"2021-02-20T13:08:45.096289+0000","last_peered":"2021-02-20T13:08:45.096289+0000","last_clean":"2021-02-20T13:08:45.096289+0000","last_became_active":"2021-02-20T13:08:45.096004+0000","last_became_peered":"2021-02-20T13:08:45.096004+0000","last_unstale":"2021-02-20T13:08:45.096289+0000","last_undegraded":"2021-02-20T13:08:45.096289+0000","last_fullsized":"2021-02-20T13:08:45.096289+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_clean_scrub_stamp":"2021-02-20T13:08:44.066399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:45.095912+0000","last_change":"2021-02-20T13:08:45.094582+0000","last_active":"2021-02-20T13:08:45.095912+0000","last_peered":"2021-02-20T13:08:45.095912+0000","last_clean":"2021-02-20T13:08:45.095912+0000","last_became_active":"2021-02-20T13:08:45.094304+0000","last_became_peered":"2021-02-20T13:08:45.094304+0000","last_unstale":"2021-02-20T13:08:45.095912+0000","last_undegraded":"2021-02-20T13:08:45.095912+0000","last_fullsized":"2021-02-20T13:08:45.095912+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_clean_scrub_stamp":"2021-02-20T13:08:44.066399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T13:08:45.100615+0000","last_change":"2021-02-20T13:08:45.099198+0000","last_active":"2021-02-20T13:08:45.100615+0000","last_peered":"2021-02-20T13:08:45.100615+0000","last_clean":"2021-02-20T13:08:45.100615+0000","last_became_active":"2021-02-20T13:08:45.098155+0000","last_became_peered":"2021-02-20T13:08:45.098155+0000","last_unstale":"2021-02-20T13:08:45.100615+0000","last_undegraded":"2021-02-20T13:08:45.100615+0000","last_fullsized":"2021-02-20T13:08:45.100615+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T13:08:44.066399+0000","last_clean_scrub_stamp":"2021-02-20T13:08:44.066399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4832,"kb_used_data":88,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94367008,"statfs":{"total":96636764160,"available":96631816192,"internally_reserved":0,"allocated":90112,"data_stored":22804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4159999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0449999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]}]},{"osd":1,"up_from":8,"seq":34359738370,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]}]}]}} 2021-02-20T13:08:50.437 INFO:tasks.ceph.ceph_manager.cluster2:clean! 2021-02-20T13:08:50.438 INFO:tasks.ceph:Waiting until ceph cluster cluster2 is healthy... 2021-02-20T13:08:50.438 INFO:tasks.ceph.ceph_manager.cluster2:wait_until_healthy 2021-02-20T13:08:50.438 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 health --format=json 2021-02-20T13:08:50.771 INFO:teuthology.orchestra.run.smithi183.stdout: 2021-02-20T13:08:50.771 INFO:teuthology.orchestra.run.smithi183.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2021-02-20T13:08:50.784 INFO:tasks.ceph.ceph_manager.cluster2:wait_until_healthy done 2021-02-20T13:08:50.785 INFO:teuthology.run_tasks:Running task exec... 2021-02-20T13:08:50.797 INFO:teuthology.task.exec:Executing custom commands... 2021-02-20T13:08:50.798 INFO:teuthology.task.exec:Running commands on role cluster1.client.mirror.0 host ubuntu@smithi183.front.sepia.ceph.com 2021-02-20T13:08:50.798 DEBUG:teuthology.orchestra.run.smithi183:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror mon '"'"'profile rbd-mirror-peer'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T13:08:51.156 INFO:teuthology.orchestra.run.smithi183.stderr:updated caps for client.mirror 2021-02-20T13:08:51.169 DEBUG:teuthology.orchestra.run.smithi183:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.0 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T13:08:51.531 INFO:teuthology.orchestra.run.smithi183.stderr:updated caps for client.mirror.0 2021-02-20T13:08:51.542 DEBUG:teuthology.orchestra.run.smithi183:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.1 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T13:08:51.891 INFO:teuthology.orchestra.run.smithi183.stderr:updated caps for client.mirror.1 2021-02-20T13:08:51.904 DEBUG:teuthology.orchestra.run.smithi183:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.2 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T13:08:52.277 INFO:teuthology.orchestra.run.smithi183.stderr:updated caps for client.mirror.2 2021-02-20T13:08:52.291 DEBUG:teuthology.orchestra.run.smithi183:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.3 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T13:08:52.664 INFO:teuthology.orchestra.run.smithi183.stderr:updated caps for client.mirror.3 2021-02-20T13:08:52.678 DEBUG:teuthology.orchestra.run.smithi183:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror mon '"'"'profile rbd-mirror-peer'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T13:08:53.054 INFO:teuthology.orchestra.run.smithi183.stderr:updated caps for client.mirror 2021-02-20T13:08:53.066 DEBUG:teuthology.orchestra.run.smithi183:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.0 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T13:08:53.445 INFO:teuthology.orchestra.run.smithi183.stderr:updated caps for client.mirror.0 2021-02-20T13:08:53.459 DEBUG:teuthology.orchestra.run.smithi183:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.1 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T13:08:53.830 INFO:teuthology.orchestra.run.smithi183.stderr:updated caps for client.mirror.1 2021-02-20T13:08:53.842 DEBUG:teuthology.orchestra.run.smithi183:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.2 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T13:08:54.214 INFO:teuthology.orchestra.run.smithi183.stderr:updated caps for client.mirror.2 2021-02-20T13:08:54.226 DEBUG:teuthology.orchestra.run.smithi183:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.3 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T13:08:54.586 INFO:teuthology.orchestra.run.smithi183.stderr:updated caps for client.mirror.3 2021-02-20T13:08:54.601 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T13:08:54.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:08:54.629 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:08:54.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:08:54.633 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T13:08:54.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T13:08:54.715 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T13:08:54.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T13:08:54.743 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T13:08:54.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T13:08:54.841 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T13:08:54.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T13:08:54.858 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T13:08:54.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T13:08:54.875 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T13:08:54.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T13:08:54.889 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T13:08:54.910 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:08:54.911 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:08:54.919 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:08:54.920 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T13:08:54.935 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:08:54.936 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:08:54.940 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:08:54.941 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T13:08:54.964 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T13:08:54.965 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T13:08:54.970 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T13:08:54.970 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T13:08:54.988 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T13:08:54.988 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T13:08:54.993 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T13:08:54.993 INFO:teuthology.run_tasks:Running task rbd-mirror-thrash... 2021-02-20T13:08:55.019 INFO:tasks.rbd_mirror_thrash:rbd_mirror_thrash using random seed: 1613826535 2021-02-20T13:08:55.019 DEBUG:tasks.rbd_mirror_thrash:Yielding 2021-02-20T13:08:55.020 INFO:teuthology.run_tasks:Running task rbd-mirror-thrash... 2021-02-20T13:08:55.039 INFO:tasks.rbd_mirror_thrash:rbd_mirror_thrash using random seed: 1613826535 2021-02-20T13:08:55.040 DEBUG:tasks.rbd_mirror_thrash:Yielding 2021-02-20T13:08:55.040 INFO:teuthology.run_tasks:Running task workunit... 2021-02-20T13:08:55.070 INFO:tasks.workunit:Pulling workunits from ref 0d796331675b92762d821b40e70f6598bcea1178 2021-02-20T13:08:55.072 INFO:tasks.workunit:Making a separate scratch dir for every client... 2021-02-20T13:08:55.072 DEBUG:teuthology.orchestra.run.smithi183:> stat -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2021-02-20T13:08:55.073 INFO:tasks.rbd_mirror_thrash:starting thrash for cluster cluster1 2021-02-20T13:08:55.073 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T13:08:55.074 INFO:tasks.rbd_mirror_thrash:starting thrash for cluster cluster2 2021-02-20T13:08:55.074 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T13:08:55.080 INFO:teuthology.orchestra.run.smithi183.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.cluster1.mirror': No such file or directory 2021-02-20T13:08:55.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:08:55.080 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2021-02-20T13:08:55.127 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.cluster1.mirror 2021-02-20T13:08:55.128 DEBUG:teuthology.orchestra.run.smithi183:> cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror && mkdir -- client.mirror 2021-02-20T13:08:55.176 INFO:tasks.workunit:timeout=6h 2021-02-20T13:08:55.176 INFO:tasks.workunit:cleanup=True 2021-02-20T13:08:55.177 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf /home/ubuntu/cephtest/clone.cluster1.client.mirror && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.cluster1.client.mirror && cd /home/ubuntu/cephtest/clone.cluster1.client.mirror && git checkout 0d796331675b92762d821b40e70f6598bcea1178 2021-02-20T13:08:55.225 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Cloning into '/home/ubuntu/cephtest/clone.cluster1.client.mirror'... 2021-02-20T13:09:26.665 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Note: checking out '0d796331675b92762d821b40e70f6598bcea1178'. 2021-02-20T13:09:26.666 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:09:26.666 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2021-02-20T13:09:26.667 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:changes and commit them, and you can discard any commits you make in this 2021-02-20T13:09:26.667 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:state without impacting any branches by performing another checkout. 2021-02-20T13:09:26.667 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:09:26.668 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:If you want to create a new branch to retain commits you create, you may 2021-02-20T13:09:26.668 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:do so (now or later) by using -b with the checkout command again. Example: 2021-02-20T13:09:26.669 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:09:26.669 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: git checkout -b 2021-02-20T13:09:26.670 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:09:26.670 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:HEAD is now at 0d79633167 cmake: libneoradostest-support should be static by dillaman · Pull Request #39587 · ceph/ceph · GitHub 2021-02-20T13:09:26.672 DEBUG:teuthology.orchestra.run.smithi183:> cd -- /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.cluster1.client.mirror 2021-02-20T13:09:26.720 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2021-02-20T13:09:26.724 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/direct_io' 2021-02-20T13:09:26.724 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2021-02-20T13:09:27.427 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2021-02-20T13:09:27.555 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2021-02-20T13:09:27.608 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/direct_io' 2021-02-20T13:09:27.610 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/fs' 2021-02-20T13:09:27.611 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2021-02-20T13:09:27.657 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/fs' 2021-02-20T13:09:27.662 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2021-02-20T13:09:27.663 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/home/ubuntu/cephtest/workunits.list.cluster1.client.mirror of=/dev/stdout 2021-02-20T13:09:27.712 INFO:tasks.workunit:Running workunits matching rbd/rbd_mirror_stress.sh on cluster1.client.mirror... 2021-02-20T13:09:27.714 INFO:tasks.workunit:Running workunit rbd/rbd_mirror_stress.sh... 2021-02-20T13:09:27.714 DEBUG:teuthology.orchestra.run.smithi183:workunit test rbd/rbd_mirror_stress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0d796331675b92762d821b40e70f6598bcea1178 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS='' MIRROR_IMAGE_MODE=snapshot MIRROR_POOL_MODE=image RBD_IMAGE_FEATURES=layering,exclusive-lock RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 RBD_MIRROR_USE_RBD_MIRROR=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh 2021-02-20T13:09:27.766 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ IMAGE_COUNT=50 2021-02-20T13:09:27.766 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ export LOCKDEP=0 2021-02-20T13:09:27.766 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ dirname /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh 2021-02-20T13:09:27.768 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ . /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_helpers.sh 2021-02-20T13:09:27.768 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ type xmlstarlet 2021-02-20T13:09:27.768 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ XMLSTARLET=xmlstarlet 2021-02-20T13:09:27.769 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ RBD_MIRROR_INSTANCES=4 2021-02-20T13:09:27.769 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CLUSTER1=cluster1 2021-02-20T13:09:27.769 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CLUSTER2=cluster2 2021-02-20T13:09:27.769 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ PEER_CLUSTER_SUFFIX= 2021-02-20T13:09:27.769 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ POOL=mirror 2021-02-20T13:09:27.770 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ PARENT_POOL=mirror_parent 2021-02-20T13:09:27.770 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ NS1=ns1 2021-02-20T13:09:27.770 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ NS2=ns2 2021-02-20T13:09:27.770 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ TEMPDIR= 2021-02-20T13:09:27.771 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ID=mirror 2021-02-20T13:09:27.771 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ RBD_IMAGE_FEATURES=layering,exclusive-lock 2021-02-20T13:09:27.771 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ MIRROR_USER_ID_PREFIX=mirror. 2021-02-20T13:09:27.771 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ MIRROR_POOL_MODE=image 2021-02-20T13:09:27.772 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ MIRROR_IMAGE_MODE=snapshot 2021-02-20T13:09:27.772 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ export CEPH_ARGS=--id mirror 2021-02-20T13:09:27.772 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ LAST_MIRROR_INSTANCE=3 2021-02-20T13:09:27.772 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ dirname /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh 2021-02-20T13:09:27.774 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ readlink -f /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/../../../src 2021-02-20T13:09:27.774 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror/src 2021-02-20T13:09:27.774 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_BIN=. 2021-02-20T13:09:27.774 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_SRC=. 2021-02-20T13:09:27.775 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -e CMakeCache.txt ] 2021-02-20T13:09:27.775 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 -gt 0 ] 2021-02-20T13:09:27.775 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ setup 2021-02-20T13:09:27.775 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local c 2021-02-20T13:09:27.776 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ trap cleanup $? INT TERM EXIT 2021-02-20T13:09:27.776 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ setup_tempdir 2021-02-20T13:09:27.776 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:09:27.776 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mktemp -d 2021-02-20T13:09:27.776 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ TEMPDIR=/tmp/tmp.CsCtSCD9Zs 2021-02-20T13:09:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -z 1 ] 2021-02-20T13:09:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ setup_pools cluster1 cluster2 2021-02-20T13:09:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local remote_cluster=cluster2 2021-02-20T13:09:27.778 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mon_map_file 2021-02-20T13:09:27.778 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mon_addr 2021-02-20T13:09:27.778 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local admin_key_file 2021-02-20T13:09:27.778 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local uuid 2021-02-20T13:09:27.779 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ARGS= ceph --cluster cluster1 osd pool create mirror 64 64 2021-02-20T13:09:28.871 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:pool 'mirror' already exists 2021-02-20T13:09:28.882 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ARGS= ceph --cluster cluster1 osd pool create mirror_parent 64 64 2021-02-20T13:09:29.877 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:pool 'mirror_parent' already exists 2021-02-20T13:09:29.887 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ARGS= rbd --cluster cluster1 pool init mirror 2021-02-20T13:09:32.822 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ARGS= rbd --cluster cluster1 pool init mirror_parent 2021-02-20T13:09:35.836 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:09:35.837 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror pool enable --site-name cluster1 mirror image 2021-02-20T13:09:35.906 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 mirror pool enable mirror_parent image 2021-02-20T13:09:35.954 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 namespace create mirror/ns1 2021-02-20T13:09:36.005 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 namespace create mirror/ns2 2021-02-20T13:09:36.052 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 mirror pool enable mirror/ns1 image 2021-02-20T13:09:36.109 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 mirror pool enable mirror/ns2 image 2021-02-20T13:09:36.166 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -z ] 2021-02-20T13:09:36.166 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -z ] 2021-02-20T13:09:36.167 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ peer_add cluster1 mirror cluster2 2021-02-20T13:09:36.167 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:36.167 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:36.168 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:09:36.168 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:36.168 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local client_cluster=cluster2 2021-02-20T13:09:36.169 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:36.169 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local remote_cluster=cluster2 2021-02-20T13:09:36.169 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local uuid_var_name 2021-02-20T13:09:36.170 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:09:36.170 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local error_code 2021-02-20T13:09:36.170 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local peer_uuid 2021-02-20T13:09:36.171 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:09:36.171 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 mirror pool peer add mirror cluster2 2021-02-20T13:09:36.215 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ peer_uuid=5248b262-5c79-4981-886a-9bf7033d425d 2021-02-20T13:09:36.216 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=0 2021-02-20T13:09:36.216 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:09:36.216 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 -eq 17 ] 2021-02-20T13:09:36.217 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 0 -eq 0 2021-02-20T13:09:36.217 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:09:36.217 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:09:36.218 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ peer_add cluster1 mirror_parent cluster2 2021-02-20T13:09:36.218 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:36.218 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:36.219 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror_parent 2021-02-20T13:09:36.219 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:36.219 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local client_cluster=cluster2 2021-02-20T13:09:36.219 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:36.220 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local remote_cluster=cluster2 2021-02-20T13:09:36.220 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local uuid_var_name 2021-02-20T13:09:36.220 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:09:36.220 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local error_code 2021-02-20T13:09:36.220 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local peer_uuid 2021-02-20T13:09:36.221 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:09:36.221 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 mirror pool peer add mirror_parent cluster2 2021-02-20T13:09:36.266 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ peer_uuid=f588bb56-1379-463a-983b-7e69a494a307 2021-02-20T13:09:36.267 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=0 2021-02-20T13:09:36.267 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:09:36.267 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 -eq 17 ] 2021-02-20T13:09:36.267 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 0 -eq 0 2021-02-20T13:09:36.267 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:09:36.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:09:36.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ setup_pools cluster2 cluster1 2021-02-20T13:09:36.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:09:36.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local remote_cluster=cluster1 2021-02-20T13:09:36.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mon_map_file 2021-02-20T13:09:36.269 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mon_addr 2021-02-20T13:09:36.269 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local admin_key_file 2021-02-20T13:09:36.269 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local uuid 2021-02-20T13:09:36.269 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ARGS= ceph --cluster cluster2 osd pool create mirror 64 64 2021-02-20T13:09:37.309 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:pool 'mirror' already exists 2021-02-20T13:09:37.319 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ARGS= ceph --cluster cluster2 osd pool create mirror_parent 64 64 2021-02-20T13:09:38.341 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:pool 'mirror_parent' already exists 2021-02-20T13:09:38.350 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ARGS= rbd --cluster cluster2 pool init mirror 2021-02-20T13:09:41.243 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ARGS= rbd --cluster cluster2 pool init mirror_parent 2021-02-20T13:09:44.260 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:09:44.261 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror pool enable --site-name cluster2 mirror image 2021-02-20T13:09:44.320 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 mirror pool enable mirror_parent image 2021-02-20T13:09:44.362 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 namespace create mirror/ns1 2021-02-20T13:09:44.409 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 namespace create mirror/ns2 2021-02-20T13:09:44.453 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 mirror pool enable mirror/ns1 image 2021-02-20T13:09:44.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 mirror pool enable mirror/ns2 image 2021-02-20T13:09:44.555 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -z ] 2021-02-20T13:09:44.555 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -z ] 2021-02-20T13:09:44.556 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ peer_add cluster2 mirror cluster1 2021-02-20T13:09:44.556 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:09:44.556 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:44.556 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:09:44.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:44.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local client_cluster=cluster1 2021-02-20T13:09:44.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:44.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local remote_cluster=cluster1 2021-02-20T13:09:44.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local uuid_var_name 2021-02-20T13:09:44.558 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:09:44.558 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local error_code 2021-02-20T13:09:44.558 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local peer_uuid 2021-02-20T13:09:44.558 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:09:44.559 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 mirror pool peer add mirror cluster1 2021-02-20T13:09:44.602 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ peer_uuid=6938f23d-c966-4bc8-a628-cb59a937238d 2021-02-20T13:09:44.603 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=0 2021-02-20T13:09:44.603 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:09:44.603 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 -eq 17 ] 2021-02-20T13:09:44.603 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 0 -eq 0 2021-02-20T13:09:44.604 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:09:44.604 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:09:44.604 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ peer_add cluster2 mirror_parent cluster1 2021-02-20T13:09:44.604 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:09:44.605 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:44.605 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror_parent 2021-02-20T13:09:44.605 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:44.605 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local client_cluster=cluster1 2021-02-20T13:09:44.605 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:44.606 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local remote_cluster=cluster1 2021-02-20T13:09:44.606 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local uuid_var_name 2021-02-20T13:09:44.606 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:09:44.607 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local error_code 2021-02-20T13:09:44.607 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local peer_uuid 2021-02-20T13:09:44.607 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:09:44.607 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 mirror pool peer add mirror_parent cluster1 2021-02-20T13:09:44.648 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ peer_uuid=160e36db-6d05-47c6-834d-165f031ee978 2021-02-20T13:09:44.648 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=0 2021-02-20T13:09:44.649 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:09:44.649 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 -eq 17 ] 2021-02-20T13:09:44.649 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 0 -eq 0 2021-02-20T13:09:44.649 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:09:44.650 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:09:44.650 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:09:44.650 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ start_mirrors cluster1 2021-02-20T13:09:44.650 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:44.650 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 0 3 2021-02-20T13:09:44.687 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ start_mirror cluster1:0 2021-02-20T13:09:44.687 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:0 2021-02-20T13:09:44.687 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:44.688 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T13:09:44.688 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:09:44.688 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:44.688 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:44.689 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:44.689 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:09:44.689 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:09:44.689 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:09:44.690 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:44.690 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:09:44.691 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:09:44.691 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -n 1 2021-02-20T13:09:44.691 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 2021-02-20T13:09:44.691 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ start_mirror cluster1:1 2021-02-20T13:09:44.692 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:1 2021-02-20T13:09:44.692 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:44.692 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T13:09:44.692 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:09:44.692 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:44.693 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:44.693 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:44.693 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:09:44.694 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:09:44.694 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:09:44.694 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:44.694 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:09:44.694 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:09:44.695 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -n 1 2021-02-20T13:09:44.695 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 2021-02-20T13:09:44.695 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ start_mirror cluster1:2 2021-02-20T13:09:44.695 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:2 2021-02-20T13:09:44.696 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:44.696 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T13:09:44.696 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:09:44.696 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:44.697 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:44.697 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:44.697 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:09:44.698 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:09:44.698 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:09:44.698 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:44.699 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:09:44.699 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:09:44.699 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -n 1 2021-02-20T13:09:44.699 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 2021-02-20T13:09:44.700 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ start_mirror cluster1:3 2021-02-20T13:09:44.700 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:3 2021-02-20T13:09:44.701 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:44.701 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T13:09:44.701 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:09:44.701 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:44.701 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:44.702 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:44.702 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:09:44.702 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:09:44.702 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:09:44.702 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:44.703 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:09:44.705 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:09:44.706 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -n 1 2021-02-20T13:09:44.707 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 2021-02-20T13:09:44.707 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ start_mirrors cluster2 2021-02-20T13:09:44.707 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:09:44.707 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 0 3 2021-02-20T13:09:44.707 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ start_mirror cluster2:0 2021-02-20T13:09:44.708 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2:0 2021-02-20T13:09:44.708 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:44.708 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster2:0 cluster instance 2021-02-20T13:09:44.708 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster2:0 2021-02-20T13:09:44.708 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:44.709 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:44.709 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster2 2021-02-20T13:09:44.709 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:09:44.709 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster2:0 ] 2021-02-20T13:09:44.710 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster2 2021-02-20T13:09:44.710 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster2 2021-02-20T13:09:44.710 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:09:44.711 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:09:44.711 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -n 1 2021-02-20T13:09:44.711 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 2021-02-20T13:09:44.712 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ start_mirror cluster2:1 2021-02-20T13:09:44.712 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2:1 2021-02-20T13:09:44.712 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:44.712 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster2:1 cluster instance 2021-02-20T13:09:44.712 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster2:1 2021-02-20T13:09:44.713 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:44.713 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:44.713 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster2 2021-02-20T13:09:44.713 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:09:44.714 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster2:1 ] 2021-02-20T13:09:44.714 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster2 2021-02-20T13:09:44.714 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster2 2021-02-20T13:09:44.714 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:09:44.714 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:09:44.715 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -n 1 2021-02-20T13:09:44.715 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 2021-02-20T13:09:44.715 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ start_mirror cluster2:2 2021-02-20T13:09:44.715 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2:2 2021-02-20T13:09:44.715 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:44.716 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster2:2 cluster instance 2021-02-20T13:09:44.716 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster2:2 2021-02-20T13:09:44.716 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:44.716 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:44.717 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster2 2021-02-20T13:09:44.717 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:09:44.719 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster2:2 ] 2021-02-20T13:09:44.719 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster2 2021-02-20T13:09:44.719 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster2 2021-02-20T13:09:44.719 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:09:44.720 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:09:44.720 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -n 1 2021-02-20T13:09:44.720 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 2021-02-20T13:09:44.720 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ start_mirror cluster2:3 2021-02-20T13:09:44.720 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2:3 2021-02-20T13:09:44.721 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:44.721 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster2:3 cluster instance 2021-02-20T13:09:44.721 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster2:3 2021-02-20T13:09:44.721 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:44.722 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:44.722 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster2 2021-02-20T13:09:44.722 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:09:44.722 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster2:3 ] 2021-02-20T13:09:44.722 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster2 2021-02-20T13:09:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster2 2021-02-20T13:09:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:09:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:09:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -n 1 2021-02-20T13:09:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 2021-02-20T13:09:44.724 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ testlog TEST: add image and test replay after client crashes 2021-02-20T13:09:44.724 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ tee -a /tmp/tmp.CsCtSCD9Zs/rbd-mirror.test.log 2021-02-20T13:09:44.724 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ date +%F %T 2021-02-20T13:09:44.724 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo 2021-02-20 13:09:44 TEST: add image and test replay after client crashes 2021-02-20T13:09:44.725 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:2021-02-20 13:09:44 TEST: add image and test replay after client crashes 2021-02-20T13:09:44.725 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=test 2021-02-20T13:09:44.725 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror test snapshot 512M 2021-02-20T13:09:44.725 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:09:44.725 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:44.726 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:09:44.726 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:44.726 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:09:44.727 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:44.727 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:09:44.727 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:09:44.727 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:44.727 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror test 512M 2021-02-20T13:09:44.728 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:09:44.728 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:44.728 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:09:44.728 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:44.728 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:09:44.729 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:44.730 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:09:44.730 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 512M ] 2021-02-20T13:09:44.730 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=512M 2021-02-20T13:09:44.730 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:44.731 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering,exclusive-lock mirror/test 2021-02-20T13:09:44.747 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:09:44.747 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror test snapshot 2021-02-20T13:09:44.747 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:09:44.748 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:09:44.748 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:09:44.748 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:09:44.748 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/test snapshot 2021-02-20T13:09:45.338 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:09:45.346 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T13:09:45.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:45.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:09:45.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:09:45.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T13:09:45.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:45.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:09:45.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:09:45.350 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state=started 2021-02-20T13:09:45.350 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:09:45.351 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:09:46.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T13:09:46.350 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:46.350 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:09:46.351 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:09:46.351 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local test_state=started 2021-02-20T13:09:46.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_result 2021-02-20T13:09:46.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_state=stopped 2021-02-20T13:09:46.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T13:09:46.353 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_instance=cluster1 2021-02-20T13:09:46.353 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:46.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:46.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ + local instance=cluster1 2021-02-20T13:09:46.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ greplocal -i loop_instance state 2021-02-20T13:09:46.355 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:09:46.378 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0 2021-02-20T13:09:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T13:09:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 0 3 2021-02-20T13:09:46.380 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T13:09:46.380 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:0 2021-02-20T13:09:46.380 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:46.381 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:46.381 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T13:09:46.382 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:09:46.382 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:46.383 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:46.383 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:46.383 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:09:46.384 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:09:46.384 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:09:46.386 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:46.386 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:09:46.387 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:09:46.387 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T13:09:46.388 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:0 2021-02-20T13:09:46.388 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:46.389 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:46.389 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T13:09:46.390 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:09:46.390 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:09:46.390 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:46.391 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:46.391 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:09:46.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:09:46.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:09:46.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:09:46.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:09:46.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:09:46.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T13:09:46.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:09:46.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:09:46.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:09:46.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T13:09:46.483 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:09:46.484 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:09:46.484 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:09:46.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T13:09:46.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:1 2021-02-20T13:09:46.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:46.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:46.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T13:09:46.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:09:46.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:46.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:46.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:46.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:09:46.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:09:46.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:09:46.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:46.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:09:46.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:09:46.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T13:09:46.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:1 2021-02-20T13:09:46.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:46.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:46.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T13:09:46.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:09:46.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:09:46.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:46.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:46.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:09:46.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:09:46.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:09:46.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:09:46.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:09:46.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:09:46.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T13:09:46.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:09:46.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:09:46.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:09:46.502 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T13:09:46.596 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:09:46.597 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:09:46.597 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:09:46.602 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T13:09:46.603 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:2 2021-02-20T13:09:46.603 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:46.603 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:46.604 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T13:09:46.604 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:09:46.605 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:46.606 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:46.606 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:46.607 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:09:46.607 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:09:46.607 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:09:46.608 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:46.608 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:09:46.609 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:09:46.609 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T13:09:46.609 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:2 2021-02-20T13:09:46.610 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:46.610 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:46.611 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T13:09:46.611 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:09:46.611 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:09:46.612 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:46.612 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:46.612 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:09:46.613 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:09:46.613 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:09:46.613 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:09:46.614 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:09:46.620 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:09:46.621 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T13:09:46.628 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:09:46.629 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:09:46.630 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:09:46.630 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T13:09:46.712 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:09:46.713 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:09:46.713 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:09:46.714 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T13:09:46.715 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:3 2021-02-20T13:09:46.715 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:46.715 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:46.715 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T13:09:46.716 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:09:46.717 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:46.717 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:46.717 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:46.718 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:09:46.718 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:09:46.718 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:09:46.718 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:46.719 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:09:46.719 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:09:46.720 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T13:09:46.720 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:3 2021-02-20T13:09:46.720 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:46.721 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:46.721 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T13:09:46.721 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:09:46.721 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:09:46.723 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:46.723 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:46.724 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:09:46.724 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:09:46.724 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:09:46.724 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:09:46.725 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:09:46.725 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:09:46.725 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T13:09:46.732 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:09:46.733 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:09:46.733 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:09:46.733 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T13:09:46.829 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:09:46.830 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:09:46.830 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:09:46.831 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:09:47.831 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T13:09:47.832 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 0 3 2021-02-20T13:09:47.835 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T13:09:47.836 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:0 2021-02-20T13:09:47.836 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:47.837 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:47.837 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T13:09:47.837 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:09:47.838 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:47.838 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:47.839 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:47.842 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:09:47.843 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:09:47.843 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:09:47.844 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:47.844 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:09:47.845 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:09:47.845 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T13:09:47.846 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:0 2021-02-20T13:09:47.846 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:47.846 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:47.847 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T13:09:47.847 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:09:47.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:09:47.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:47.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:47.849 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:09:47.850 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:09:47.850 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:09:47.850 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:09:47.851 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:09:47.851 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:09:47.852 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T13:09:47.855 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:09:47.856 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:09:47.856 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:09:47.856 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T13:09:47.958 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:09:47.958 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:09:47.959 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:09:47.960 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T13:09:47.961 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:1 2021-02-20T13:09:47.961 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:47.961 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:47.961 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T13:09:47.962 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:09:47.962 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:47.962 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:47.962 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:47.963 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:09:47.963 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:09:47.964 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:09:47.964 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:47.964 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:09:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:09:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T13:09:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:1 2021-02-20T13:09:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:47.966 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T13:09:47.966 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:09:47.966 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:09:47.966 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:47.966 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:47.967 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:09:47.967 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:09:47.967 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:09:47.967 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:09:47.967 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:09:47.968 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:09:47.968 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T13:09:47.981 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:09:47.981 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:09:47.982 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:09:47.982 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T13:09:48.079 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:09:48.079 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:09:48.080 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:09:48.083 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T13:09:48.083 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:2 2021-02-20T13:09:48.084 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:48.084 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:48.085 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T13:09:48.085 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:09:48.085 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:48.086 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:48.086 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:48.086 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:09:48.087 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:09:48.087 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:09:48.087 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:48.088 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:09:48.088 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:09:48.088 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T13:09:48.088 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:2 2021-02-20T13:09:48.089 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:48.089 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:48.090 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T13:09:48.090 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:09:48.091 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:09:48.091 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:48.092 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:48.092 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:09:48.092 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:09:48.093 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:09:48.094 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:09:48.094 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:09:48.095 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:09:48.095 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T13:09:48.102 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:09:48.103 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:09:48.103 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:09:48.103 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T13:09:48.198 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:09:48.198 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:09:48.198 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:09:48.199 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T13:09:48.199 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:3 2021-02-20T13:09:48.200 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:48.200 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:48.200 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T13:09:48.201 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:09:48.201 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:48.201 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:48.202 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:48.202 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:09:48.202 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:09:48.203 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:09:48.203 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:48.204 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:09:48.204 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:09:48.204 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T13:09:48.205 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:3 2021-02-20T13:09:48.205 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:48.205 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:48.206 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T13:09:48.206 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:09:48.207 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:09:48.207 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:48.208 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:48.208 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:09:48.208 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:09:48.209 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:09:48.209 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:09:48.210 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:09:48.210 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:09:48.211 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T13:09:48.219 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:09:48.219 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:09:48.220 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:09:48.220 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T13:09:48.314 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:09:48.315 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:09:48.315 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:09:48.316 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 2 2021-02-20T13:09:50.316 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T13:09:50.316 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 0 3 2021-02-20T13:09:50.318 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T13:09:50.319 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:0 2021-02-20T13:09:50.319 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:50.319 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:50.319 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T13:09:50.320 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:09:50.320 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:50.320 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:50.320 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:50.321 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:09:50.321 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:09:50.321 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:09:50.323 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:50.324 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:09:50.324 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:09:50.324 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T13:09:50.324 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:0 2021-02-20T13:09:50.325 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:50.325 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:50.325 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T13:09:50.325 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:09:50.326 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:09:50.326 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:50.326 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:50.326 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:09:50.327 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:09:50.327 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:09:50.327 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:09:50.327 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:09:50.328 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:09:50.329 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T13:09:50.337 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:09:50.337 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:09:50.337 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:09:50.338 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T13:09:50.430 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:09:50.430 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:09:50.430 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:09:50.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T13:09:50.432 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:1 2021-02-20T13:09:50.432 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:50.432 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:50.432 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T13:09:50.433 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:09:50.433 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:50.433 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:50.433 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:50.433 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:09:50.434 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:09:50.434 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:09:50.434 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:50.434 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:09:50.434 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:09:50.435 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T13:09:50.436 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:1 2021-02-20T13:09:50.436 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:50.436 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:50.437 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T13:09:50.437 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:09:50.437 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:09:50.437 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:50.438 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:50.438 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:09:50.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:09:50.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:09:50.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:09:50.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:09:50.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:09:50.441 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T13:09:50.452 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:09:50.453 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:09:50.453 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:09:50.453 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T13:09:50.544 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:09:50.544 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:09:50.544 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:09:50.546 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T13:09:50.546 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:2 2021-02-20T13:09:50.546 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:50.547 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:50.547 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T13:09:50.547 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:09:50.547 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:50.547 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:50.548 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:50.548 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:09:50.548 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:09:50.548 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:09:50.548 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:50.549 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:09:50.549 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:09:50.549 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T13:09:50.549 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:2 2021-02-20T13:09:50.549 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:50.550 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:50.550 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T13:09:50.550 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:09:50.550 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:09:50.552 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:50.552 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:50.553 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:09:50.553 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:09:50.553 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:09:50.553 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:09:50.554 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:09:50.554 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:09:50.554 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T13:09:50.567 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:09:50.567 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:09:50.568 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:09:50.568 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T13:09:50.661 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:09:50.662 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:09:50.662 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:09:50.665 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T13:09:50.665 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:3 2021-02-20T13:09:50.665 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:50.666 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:50.666 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T13:09:50.666 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:09:50.666 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:50.667 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:50.667 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:50.667 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:09:50.667 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:09:50.668 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:09:50.668 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:50.668 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:09:50.668 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:09:50.669 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T13:09:50.669 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:3 2021-02-20T13:09:50.669 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:50.669 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:50.670 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T13:09:50.670 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:09:50.671 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:09:50.671 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:50.672 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:50.672 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:09:50.672 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:09:50.673 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:09:50.673 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:09:50.673 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:09:50.673 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:09:50.673 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T13:09:50.683 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:09:50.683 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:09:50.683 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:09:50.684 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T13:09:50.777 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:09:50.778 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:09:50.778 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:09:50.778 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 4 2021-02-20T13:09:54.780 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T13:09:54.780 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 0 3 2021-02-20T13:09:54.782 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T13:09:54.782 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:0 2021-02-20T13:09:54.782 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:54.782 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:54.783 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T13:09:54.783 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:09:54.783 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:54.783 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:54.784 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:54.784 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:09:54.784 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:09:54.784 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:09:54.784 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:54.785 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:09:54.785 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:09:54.785 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T13:09:54.785 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:0 2021-02-20T13:09:54.785 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:54.786 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:54.786 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T13:09:54.786 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:09:54.786 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:09:54.786 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:54.787 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:54.787 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:09:54.787 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:09:54.787 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:09:54.787 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:09:54.788 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:09:54.788 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:09:54.788 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T13:09:54.800 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:09:54.800 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:09:54.801 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:09:54.801 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T13:09:54.909 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:09:54.909 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:09:54.910 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:09:54.911 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T13:09:54.911 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:1 2021-02-20T13:09:54.911 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:54.912 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:54.912 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T13:09:54.912 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:09:54.912 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:54.912 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:54.913 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:54.913 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:09:54.913 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:09:54.913 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:09:54.914 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:54.914 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:09:54.914 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:09:54.915 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T13:09:54.919 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:1 2021-02-20T13:09:54.919 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:54.919 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:54.919 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T13:09:54.920 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:09:54.920 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:09:54.920 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:54.920 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:54.920 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:09:54.921 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:09:54.921 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:09:54.921 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:09:54.922 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:09:54.922 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:09:54.922 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T13:09:54.927 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:09:54.927 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:09:54.928 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:09:54.928 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T13:09:55.042 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:09:55.042 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:09:55.043 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:09:55.044 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T13:09:55.045 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:2 2021-02-20T13:09:55.045 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:55.045 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:55.045 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T13:09:55.046 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:09:55.046 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:55.046 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:55.046 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:55.046 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:09:55.047 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:09:55.047 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:09:55.047 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:55.047 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:09:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:09:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T13:09:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:2 2021-02-20T13:09:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T13:09:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:09:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:09:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:55.050 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:09:55.050 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:09:55.050 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:09:55.050 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:09:55.051 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:09:55.051 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:09:55.051 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T13:09:55.067 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:09:55.067 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:09:55.069 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:09:55.069 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T13:09:55.206 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:09:55.206 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:09:55.206 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:09:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T13:09:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:3 2021-02-20T13:09:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:09:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T13:09:55.210 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:09:55.210 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:09:55.211 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:55.211 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:55.211 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:09:55.212 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:09:55.212 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:09:55.212 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:55.212 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:09:55.212 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:09:55.213 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T13:09:55.213 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:3 2021-02-20T13:09:55.213 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:09:55.213 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:09:55.213 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T13:09:55.214 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:09:55.214 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:09:55.214 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:09:55.214 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:09:55.214 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:09:55.215 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:09:55.215 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:09:55.215 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:09:55.215 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:09:55.215 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:09:55.216 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T13:09:55.229 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:09:55.229 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:09:55.229 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:09:55.229 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T13:09:55.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:09:55.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:09:55.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:09:55.355 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 8 2021-02-20T13:09:58.074 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6027967451584053) > weight (0.25) 2021-02-20T13:09:58.074 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6476920374019158) > weight (0.25) 2021-02-20T13:09:58.075 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T13:09:58.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T13:09:58.076 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T13:09:58.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T13:09:58.072+0000 7f84eb22e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 19157) UID: 1000 2021-02-20T13:09:58.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:10:03.078 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T13:10:03.084 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:10:03.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:10:03.086 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T13:10:03.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T13:10:03.087 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T13:10:03.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T13:10:03.108 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T13:10:03.356 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T13:10:03.356 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 0 3 2021-02-20T13:10:03.359 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T13:10:03.360 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:0 2021-02-20T13:10:03.360 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:10:03.360 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:10:03.361 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T13:10:03.361 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:10:03.362 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:10:03.362 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:10:03.362 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:10:03.363 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:10:03.363 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:10:03.364 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:10:03.369 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:10:03.369 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:10:03.370 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:10:03.370 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T13:10:03.371 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:0 2021-02-20T13:10:03.371 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:10:03.371 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:10:03.372 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T13:10:03.372 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:10:03.373 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:10:03.373 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:10:03.373 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:10:03.374 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:10:03.374 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:10:03.375 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:10:03.375 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:10:03.375 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:10:03.376 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:10:03.376 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T13:10:03.378 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:10:03.378 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:10:03.379 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:10:03.379 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T13:10:03.482 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:10:03.484 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ status_result= "state": "Replaying" 2021-02-20T13:10:03.484 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo "state": "Replaying" 2021-02-20T13:10:03.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep -i Replaying 2021-02-20T13:10:03.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: "state": "Replaying" 2021-02-20T13:10:03.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_state=started 2021-02-20T13:10:03.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test started = started 2021-02-20T13:10:03.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:10:03.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ clean_snap_name= 2021-02-20T13:10:03.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 1 10 2021-02-20T13:10:03.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T13:10:03.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:10:03.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:03.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:03.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:10:03.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=9 2021-02-20T13:10:03.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:10:03.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T13:10:03.831 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 48189440 bytes 2021-02-20T13:10:04.082 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 99176448 bytes 2021-02-20T13:10:04.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 164264960 bytes 2021-02-20T13:10:04.572 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 187788288 bytes 2021-02-20T13:10:04.949 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 274125824 bytes 2021-02-20T13:10:05.257 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 332217344 bytes 2021-02-20T13:10:05.645 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 408866816 bytes 2021-02-20T13:10:05.895 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 442522624 bytes 2021-02-20T13:10:06.130 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 479891456 bytes 2021-02-20T13:10:06.526 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 568300544 bytes 2021-02-20T13:10:06.648 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 582193152 bytes 2021-02-20T13:10:06.761 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 590854144 bytes 2021-02-20T13:10:06.928 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 607785984 bytes 2021-02-20T13:10:07.137 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 638414848 bytes 2021-02-20T13:10:07.539 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 724948992 bytes 2021-02-20T13:10:07.694 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 745645056 bytes 2021-02-20T13:10:07.994 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 789628928 bytes 2021-02-20T13:10:08.155 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 809669632 bytes 2021-02-20T13:10:08.476 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 869204992 bytes 2021-02-20T13:10:08.779 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 919106560 bytes 2021-02-20T13:10:09.068 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 975279104 bytes 2021-02-20T13:10:09.416 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 1037465600 bytes 2021-02-20T13:10:09.677 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 1080176640 bytes 2021-02-20T13:10:10.144 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 1157026816 bytes 2021-02-20T13:10:10.340 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 1184500736 bytes 2021-02-20T13:10:10.618 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 1222397952 bytes 2021-02-20T13:10:10.719 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 1240314880 bytes 2021-02-20T13:10:11.029 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2800 IOs, 1299910656 bytes 2021-02-20T13:10:11.459 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2900 IOs, 1363067904 bytes 2021-02-20T13:10:11.845 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3000 IOs, 1419863040 bytes 2021-02-20T13:10:12.040 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3100 IOs, 1460791296 bytes 2021-02-20T13:10:12.382 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3200 IOs, 1501727744 bytes 2021-02-20T13:10:12.498 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:10:12.498 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:10:12.499 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:10:12.499 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:10:12.499 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T13:10:12.499 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:10:12.499 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:10:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local description_pattern= 2021-02-20T13:10:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local service_pattern= 2021-02-20T13:10:12.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:10:13.499 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T13:10:13.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:10:13.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:13.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:13.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:10:13.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local description_pattern= 2021-02-20T13:10:13.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local service_pattern= 2021-02-20T13:10:13.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T13:10:13.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T13:10:13.502 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:10:13.502 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:10:13.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T13:10:13.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ tee /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:10:13.574 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:test: 2021-02-20T13:10:13.574 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: global_id: 49a0b5eb-7d69-4824-806b-1ece12354eb9 2021-02-20T13:10:13.574 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state: up+replaying 2021-02-20T13:10:13.574 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"remote_snapshot_timestamp":1613826585,"replay_state":"idle"} 2021-02-20T13:10:13.575 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: service: mirror.0 on smithi183 2021-02-20T13:10:13.575 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: last_update: 2021-02-20 13:09:56 2021-02-20T13:10:13.575 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: peer_sites: 2021-02-20T13:10:13.575 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: name: cluster2 2021-02-20T13:10:13.576 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state: up+stopped 2021-02-20T13:10:13.576 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: description: local image is primary 2021-02-20T13:10:13.576 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: last_update: 2021-02-20 13:09:57 2021-02-20T13:10:13.582 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:10:13.583 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: state: up+replaying 2021-02-20T13:10:13.584 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^ description: .* /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:10:13.585 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"remote_snapshot_timestamp":1613826585,"replay_state":"idle"} 2021-02-20T13:10:13.586 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:10:13.586 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo up+replaying 2021-02-20T13:10:13.587 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^up+ 2021-02-20T13:10:13.587 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:up+replaying 2021-02-20T13:10:13.587 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep service: *mirror..* on /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:10:13.588 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: service: mirror.0 on smithi183 2021-02-20T13:10:13.588 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:10:13.591 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local last_update=2021-02-20 13:09:56 2021-02-20T13:10:13.591 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 13:09:56 2021-02-20T13:10:13.592 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:10:13.592 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:13.592 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:13.594 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:10:13.594 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local prev_last_update=2021-02-20 13:09:56 2021-02-20T13:10:13.594 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T13:10:13.594 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T13:10:13.595 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:10:13.595 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror.mirror_status 2021-02-20T13:10:13.596 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T13:10:13.638 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T13:10:13.682 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local last_update state 2021-02-20T13:10:13.682 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T13:10:13.729 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ last_update=2021-02-20 13:09:56 2021-02-20T13:10:13.729 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T13:10:13.735 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ state=up+replaying 2021-02-20T13:10:13.735 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep up+replaying 2021-02-20T13:10:13.735 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo up+replaying 2021-02-20T13:10:13.737 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:up+replaying 2021-02-20T13:10:13.737 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:10:13.738 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:10:13.741 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap1 2021-02-20T13:10:13.742 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_snap cluster2 mirror test snap1 2021-02-20T13:10:13.742 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:10:13.742 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:13.742 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:13.743 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap1 2021-02-20T13:10:13.743 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap1 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:10:40.188 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T13:10:40.189 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:10:40.190 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:40.190 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:40.190 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T13:10:40.191 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:10:40.191 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:40.192 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:40.192 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state=started 2021-02-20T13:10:40.193 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:10:40.194 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:10:41.189 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T13:10:41.190 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:10:41.190 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:41.190 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:41.190 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local test_state=started 2021-02-20T13:10:41.190 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_result 2021-02-20T13:10:41.191 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_state=stopped 2021-02-20T13:10:41.191 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T13:10:41.191 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_instance=cluster1 2021-02-20T13:10:41.191 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ + shift 2021-02-20T13:10:41.192 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:grep -i state 2021-02-20T13:10:41.192 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:10:41.192 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance=cluster1 2021-02-20T13:10:41.192 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loop_instance 2021-02-20T13:10:41.192 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0 2021-02-20T13:10:41.194 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T13:10:41.194 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 0 3 2021-02-20T13:10:41.194 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T13:10:41.195 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:0 2021-02-20T13:10:41.195 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:10:41.195 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:10:41.195 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T13:10:41.195 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:10:41.196 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:10:41.196 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:10:41.196 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:10:41.196 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:10:41.196 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:10:41.197 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:10:41.197 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:10:41.197 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:10:41.198 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:10:41.198 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T13:10:41.198 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:0 2021-02-20T13:10:41.199 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:10:41.200 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:10:41.200 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T13:10:41.201 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:10:41.201 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:10:41.201 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:10:41.201 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:10:41.202 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:10:41.202 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:10:41.203 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:10:41.203 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:10:41.204 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:10:41.204 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:10:41.204 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T13:10:41.215 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:10:41.215 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:10:41.215 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:10:41.216 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T13:10:41.329 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:10:41.330 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ status_result= "state": "Replaying" 2021-02-20T13:10:41.331 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo "state": "Replaying" 2021-02-20T13:10:41.331 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep -i Replaying 2021-02-20T13:10:41.334 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: "state": "Replaying" 2021-02-20T13:10:41.334 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_state=started 2021-02-20T13:10:41.335 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test started = started 2021-02-20T13:10:41.335 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:10:41.335 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T13:10:41.335 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:10:41.335 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:10:41.336 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:41.336 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:41.336 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = journal ] 2021-02-20T13:10:41.336 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = snapshot ] 2021-02-20T13:10:41.337 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T13:10:41.337 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:10:41.337 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:10:41.337 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:41.337 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:41.338 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T13:10:41.338 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster2-mirror-test.status 2021-02-20T13:10:41.338 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:10:41.339 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:10:41.339 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T13:10:41.340 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror-test.status 2021-02-20T13:10:41.340 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:10:41.341 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:10:41.341 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T13:10:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:10:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T13:10:42.199 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Snapshot ID: 6 2021-02-20T13:10:42.211 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:10:42.212 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:10:42.212 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:42.212 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:42.212 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:10:42.213 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T13:10:42.213 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:10:42.265 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T13:10:42.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snapshot_id=6 2021-02-20T13:10:42.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:10:42.269 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.2 2021-02-20T13:10:42.470 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:10:42.470 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:10:42.470 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:42.470 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:42.471 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:10:42.471 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:10:42.471 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:10:42.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:10:42.540 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id=4 2021-02-20T13:10:42.540 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 6 = 4 2021-02-20T13:10:42.540 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.4 2021-02-20T13:10:42.942 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:10:42.942 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:10:42.942 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:42.943 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:42.943 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:10:42.943 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:10:42.943 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:10:43.010 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:10:43.010 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:10:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:10:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:10:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:10:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:10:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 6 = 2021-02-20T13:10:43.016 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.8 2021-02-20T13:10:43.816 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:10:43.816 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:10:43.817 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:43.817 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:43.817 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:10:43.817 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:10:43.818 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:10:43.882 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:10:43.883 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:10:43.887 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:10:43.887 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:10:43.888 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:10:43.888 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:10:43.889 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 6 = 2021-02-20T13:10:43.889 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1.6 2021-02-20T13:10:45.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:10:45.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:10:45.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:45.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:45.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:10:45.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:10:45.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:10:45.551 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:10:45.556 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id=6 2021-02-20T13:10:45.556 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 6 = 6 2021-02-20T13:10:45.556 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:10:45.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snap_present cluster1 mirror test snap1 2021-02-20T13:10:45.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:10:45.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:45.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:45.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap1 2021-02-20T13:10:45.558 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:10:45.558 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:10:46.558 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 info mirror/test@snap1 2021-02-20T13:10:46.622 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:rbd image 'test': 2021-02-20T13:10:46.622 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: size 512 MiB in 128 objects 2021-02-20T13:10:46.623 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: order 22 (4 MiB objects) 2021-02-20T13:10:46.623 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: snapshot_count: 3 2021-02-20T13:10:46.623 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: id: 109d36d23f3 2021-02-20T13:10:46.623 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: block_name_prefix: rbd_data.109d36d23f3 2021-02-20T13:10:46.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: format: 2 2021-02-20T13:10:46.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: features: layering, exclusive-lock, operations, non-primary 2021-02-20T13:10:46.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: op_features: snap-trash 2021-02-20T13:10:46.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: flags: 2021-02-20T13:10:46.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: create_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:10:46.625 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: access_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:10:46.625 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: modify_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:10:46.625 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: protected: False 2021-02-20T13:10:46.625 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring state: enabled 2021-02-20T13:10:46.625 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring mode: snapshot 2021-02-20T13:10:46.626 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring global id: 49a0b5eb-7d69-4824-806b-1ece12354eb9 2021-02-20T13:10:46.626 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring primary: false 2021-02-20T13:10:46.628 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:10:46.628 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:10:46.628 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap1 2021-02-20T13:10:46.628 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:46.629 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:46.629 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap1 2021-02-20T13:10:46.629 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:10:46.629 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:10:46.629 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:10:46.630 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:10:46.630 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1 /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:10:47.075 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5045785714224758) > weight (0.25) 2021-02-20T13:10:47.076 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9042782281791478) > weight (0.25) 2021-02-20T13:10:47.076 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T13:10:47.076 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T13:10:47.077 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T13:10:47.077 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T13:10:47.076+0000 7f888a8dc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 19212) UID: 1000 2021-02-20T13:10:47.280 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:10:47.421 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:10:47.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1 /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:10:48.078 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T13:10:48.078 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:10:48.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:10:48.079 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T13:10:48.079 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T13:10:48.080 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T13:10:48.082 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T13:10:48.082 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T13:10:48.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:10:48.443 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:10:48.720 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:10:48.847 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:10:48.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ clean_snap_name=snap1-clean 2021-02-20T13:10:48.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_snap cluster2 mirror test snap1-clean 2021-02-20T13:10:48.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:10:48.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:48.849 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:48.849 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap1-clean 2021-02-20T13:10:48.849 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap1-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T13:10:49.227 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T13:10:49.228 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:10:49.228 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:49.228 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:49.228 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:10:49.229 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=9 2021-02-20T13:10:49.229 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:10:49.230 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T13:10:50.240 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 47474688 bytes 2021-02-20T13:10:50.826 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 97660928 bytes 2021-02-20T13:10:51.089 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 132709376 bytes 2021-02-20T13:10:51.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 154743808 bytes 2021-02-20T13:10:51.829 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 220526592 bytes 2021-02-20T13:10:52.640 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 367348736 bytes 2021-02-20T13:10:52.826 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 390401024 bytes 2021-02-20T13:10:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 391037952 bytes 2021-02-20T13:10:53.225 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 455349248 bytes 2021-02-20T13:10:53.871 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 563382272 bytes 2021-02-20T13:10:54.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 667405312 bytes 2021-02-20T13:10:54.770 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 710706176 bytes 2021-02-20T13:10:55.102 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 748126208 bytes 2021-02-20T13:10:55.240 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 787331072 bytes 2021-02-20T13:10:55.651 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 829837312 bytes 2021-02-20T13:10:56.258 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 934644736 bytes 2021-02-20T13:10:56.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 964140032 bytes 2021-02-20T13:10:56.729 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 1006962688 bytes 2021-02-20T13:10:56.997 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 1042585600 bytes 2021-02-20T13:10:57.445 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 1124741120 bytes 2021-02-20T13:10:57.717 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 1161330688 bytes 2021-02-20T13:10:57.845 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 1172753408 bytes 2021-02-20T13:10:58.079 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 1218479104 bytes 2021-02-20T13:10:58.233 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:10:58.234 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:10:58.234 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:10:58.234 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:10:58.234 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T13:10:58.234 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:10:58.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:58.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:58.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:10:58.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local description_pattern= 2021-02-20T13:10:58.236 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local service_pattern= 2021-02-20T13:10:58.236 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:10:59.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T13:10:59.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:10:59.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:59.236 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:59.236 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:10:59.236 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local description_pattern= 2021-02-20T13:10:59.236 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local service_pattern= 2021-02-20T13:10:59.237 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T13:10:59.237 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T13:10:59.237 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:10:59.238 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:10:59.238 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T13:10:59.239 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ tee /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:10:59.304 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:test: 2021-02-20T13:10:59.304 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: global_id: 49a0b5eb-7d69-4824-806b-1ece12354eb9 2021-02-20T13:10:59.305 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state: up+replaying 2021-02-20T13:10:59.305 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: description: replaying, {"bytes_per_second":17166677.33,"bytes_per_snapshot":257500160.0,"local_snapshot_timestamp":1613826642,"remote_snapshot_timestamp":1613826642,"replay_state":"idle"} 2021-02-20T13:10:59.305 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: service: mirror.0 on smithi183 2021-02-20T13:10:59.306 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: last_update: 2021-02-20 13:10:54 2021-02-20T13:10:59.306 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: peer_sites: 2021-02-20T13:10:59.306 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: name: cluster2 2021-02-20T13:10:59.306 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state: up+stopped 2021-02-20T13:10:59.307 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: description: local image is primary 2021-02-20T13:10:59.307 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: last_update: 2021-02-20 13:10:55 2021-02-20T13:10:59.312 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:10:59.314 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: state: up+replaying 2021-02-20T13:10:59.314 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^ description: .* /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:10:59.316 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: description: replaying, {"bytes_per_second":17166677.33,"bytes_per_snapshot":257500160.0,"local_snapshot_timestamp":1613826642,"remote_snapshot_timestamp":1613826642,"replay_state":"idle"} 2021-02-20T13:10:59.316 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:10:59.317 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo up+replaying 2021-02-20T13:10:59.317 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^up+ 2021-02-20T13:10:59.318 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:up+replaying 2021-02-20T13:10:59.318 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep service: *mirror..* on /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:10:59.319 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: service: mirror.0 on smithi183 2021-02-20T13:10:59.320 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:10:59.323 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local last_update=2021-02-20 13:10:54 2021-02-20T13:10:59.323 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 13:10:54 2021-02-20T13:10:59.323 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:10:59.324 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:59.324 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:59.324 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:10:59.326 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local prev_last_update=2021-02-20 13:10:54 2021-02-20T13:10:59.326 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T13:10:59.327 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T13:10:59.327 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:10:59.328 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror.mirror_status 2021-02-20T13:10:59.329 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T13:10:59.369 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T13:10:59.413 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local last_update state 2021-02-20T13:10:59.414 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T13:10:59.419 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ last_update=2021-02-20 13:10:54 2021-02-20T13:10:59.419 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T13:10:59.424 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ state=up+replaying 2021-02-20T13:10:59.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo up+replaying 2021-02-20T13:10:59.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep up+replaying 2021-02-20T13:10:59.428 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:up+replaying 2021-02-20T13:10:59.428 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:10:59.429 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:10:59.430 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap2 2021-02-20T13:10:59.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_snap cluster2 mirror test snap2 2021-02-20T13:10:59.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:10:59.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:10:59.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:10:59.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap2 2021-02-20T13:10:59.432 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap2 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:11:04.109 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3503814505459496) > weight (0.25) 2021-02-20T13:11:04.111 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8886416945709177) > weight (0.25) 2021-02-20T13:11:04.111 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9919820343290019) > weight (0.25) 2021-02-20T13:11:04.111 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5647286798811418) > weight (0.25) 2021-02-20T13:11:04.112 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T13:11:29.141 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T13:11:29.141 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:11:29.142 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:29.142 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:29.143 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T13:11:29.143 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:11:29.143 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:29.148 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:29.149 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state=started 2021-02-20T13:11:29.149 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:11:29.150 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:11:30.141 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T13:11:30.142 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:11:30.142 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:30.142 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:30.143 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local test_state=started 2021-02-20T13:11:30.143 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_result 2021-02-20T13:11:30.144 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_state=stopped 2021-02-20T13:11:30.144 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T13:11:30.144 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_instance=cluster1 2021-02-20T13:11:30.144 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:11:30.145 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local+ cluster=cluster1 2021-02-20T13:11:30.145 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance=cluster1 2021-02-20T13:11:30.145 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loop_instance 2021-02-20T13:11:30.145 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:grep -i state 2021-02-20T13:11:30.146 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0 2021-02-20T13:11:30.152 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T13:11:30.153 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 0 3 2021-02-20T13:11:30.155 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T13:11:30.156 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:0 2021-02-20T13:11:30.156 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:11:30.157 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:11:30.157 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T13:11:30.157 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:11:30.157 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:11:30.158 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:11:30.158 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:11:30.158 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:11:30.158 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:11:30.159 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:11:30.160 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:11:30.160 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:11:30.160 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:11:30.160 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T13:11:30.162 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:0 2021-02-20T13:11:30.163 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:11:30.163 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:11:30.164 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T13:11:30.164 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:11:30.164 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:11:30.164 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:11:30.165 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:11:30.165 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:11:30.165 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:11:30.165 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:11:30.166 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:11:30.167 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:11:30.167 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:11:30.168 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T13:11:30.168 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:11:30.168 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:11:30.169 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:11:30.169 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T13:11:30.288 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:11:30.288 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ status_result= "state": "Replaying" 2021-02-20T13:11:30.288 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo "state": "Replaying" 2021-02-20T13:11:30.288 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep -i Replaying 2021-02-20T13:11:30.291 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: "state": "Replaying" 2021-02-20T13:11:30.291 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_state=started 2021-02-20T13:11:30.292 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test started = started 2021-02-20T13:11:30.292 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:11:30.292 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T13:11:30.292 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:11:30.292 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:11:30.293 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:30.293 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:30.293 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = journal ] 2021-02-20T13:11:30.293 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = snapshot ] 2021-02-20T13:11:30.294 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T13:11:30.294 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:11:30.294 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:11:30.294 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:30.294 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:30.295 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T13:11:30.295 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster2-mirror-test.status 2021-02-20T13:11:30.295 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:11:30.295 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:11:30.296 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T13:11:30.296 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror-test.status 2021-02-20T13:11:30.296 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:11:30.297 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:11:30.297 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T13:11:30.297 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:11:30.298 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:30.298 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:30.298 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T13:11:31.149 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Snapshot ID: 10 2021-02-20T13:11:31.162 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:11:31.163 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:11:31.163 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:31.164 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:31.164 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:11:31.165 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T13:11:31.165 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:11:31.221 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T13:11:31.222 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snapshot_id=10 2021-02-20T13:11:31.222 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:11:31.223 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.2 2021-02-20T13:11:31.430 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:11:31.430 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:11:31.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:31.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:31.432 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:11:31.432 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:11:31.432 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:11:31.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:11:31.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id=6 2021-02-20T13:11:31.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 10 = 6 2021-02-20T13:11:31.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.4 2021-02-20T13:11:31.903 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:11:31.903 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:11:31.903 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:31.903 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:31.904 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:11:31.904 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:11:31.904 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:11:31.970 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:11:31.970 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:11:31.974 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:11:31.975 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:11:31.975 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:11:31.976 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:11:31.977 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 10 = 2021-02-20T13:11:31.977 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.8 2021-02-20T13:11:32.777 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:11:32.777 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:11:32.778 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:32.778 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:32.778 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:11:32.778 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:11:32.779 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:11:32.843 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:11:32.843 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:11:32.847 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:11:32.847 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:11:32.847 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:11:32.847 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:11:32.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 10 = 2021-02-20T13:11:32.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1.6 2021-02-20T13:11:34.454 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:11:34.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:11:34.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:34.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:34.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:11:34.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:11:34.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:11:34.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:11:34.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:11:34.529 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:11:34.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:11:34.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:11:34.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:11:34.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 10 = 2021-02-20T13:11:34.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 2 2021-02-20T13:11:36.523 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:11:36.524 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:11:36.524 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:36.525 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:36.525 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:11:36.526 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:11:36.526 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:11:36.579 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:11:36.580 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:11:36.584 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:11:36.584 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:11:36.585 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:11:36.585 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:11:36.586 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 10 = 2021-02-20T13:11:36.586 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 2 2021-02-20T13:11:38.586 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:11:38.586 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:11:38.586 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:38.586 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:38.587 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:11:38.587 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:11:38.587 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:11:38.651 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:11:38.656 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id=10 2021-02-20T13:11:38.657 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 10 = 10 2021-02-20T13:11:38.657 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:11:38.657 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snap_present cluster1 mirror test snap2 2021-02-20T13:11:38.657 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:11:38.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:38.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:38.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap2 2021-02-20T13:11:38.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:11:38.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:11:39.657 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 info mirror/test@snap2 2021-02-20T13:11:39.720 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:rbd image 'test': 2021-02-20T13:11:39.720 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: size 512 MiB in 128 objects 2021-02-20T13:11:39.720 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: order 22 (4 MiB objects) 2021-02-20T13:11:39.721 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: snapshot_count: 4 2021-02-20T13:11:39.721 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: id: 109d36d23f3 2021-02-20T13:11:39.721 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: block_name_prefix: rbd_data.109d36d23f3 2021-02-20T13:11:39.721 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: format: 2 2021-02-20T13:11:39.721 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: features: layering, exclusive-lock, non-primary 2021-02-20T13:11:39.722 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: op_features: 2021-02-20T13:11:39.722 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: flags: 2021-02-20T13:11:39.722 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: create_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:11:39.722 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: access_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:11:39.722 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: modify_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:11:39.723 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: protected: False 2021-02-20T13:11:39.723 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring state: enabled 2021-02-20T13:11:39.723 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring mode: snapshot 2021-02-20T13:11:39.723 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring global id: 49a0b5eb-7d69-4824-806b-1ece12354eb9 2021-02-20T13:11:39.723 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring primary: false 2021-02-20T13:11:39.726 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:11:39.726 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snap1-clean ] 2021-02-20T13:11:39.726 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap1-clean 2021-02-20T13:11:39.726 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:39.727 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:39.727 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap1-clean 2021-02-20T13:11:39.727 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:11:39.727 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:11:39.728 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:11:39.728 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:11:39.728 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1-clean /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:11:40.558 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:11:40.562 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1-clean /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:11:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:11:41.623 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:11:41.902 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:11:42.030 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:11:42.030 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap2 2021-02-20T13:11:42.031 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:42.031 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:42.031 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap2 2021-02-20T13:11:42.032 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:11:42.032 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:11:42.033 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:11:42.033 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:11:42.035 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2 /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:11:42.681 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T13:11:42.676+0000 7f4a0fb5a700 -1 bdev(0x55fc57f52000 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 1 2021-02-20T13:11:42.787 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:11:42.791 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2 /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:11:43.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:11:43.662 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:11:43.943 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:11:44.070 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:11:44.071 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ clean_snap_name=snap2-clean 2021-02-20T13:11:44.071 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_snap cluster2 mirror test snap2-clean 2021-02-20T13:11:44.071 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:11:44.071 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:44.071 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:44.072 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap2-clean 2021-02-20T13:11:44.072 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap2-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T13:11:44.177 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T13:11:44.178 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:11:44.178 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:44.178 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:44.179 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:11:44.179 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=12 2021-02-20T13:11:44.180 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:11:44.180 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T13:11:45.034 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 22682624 bytes 2021-02-20T13:11:45.761 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 74382336 bytes 2021-02-20T13:11:46.181 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 110279680 bytes 2021-02-20T13:11:46.600 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 164852736 bytes 2021-02-20T13:11:46.916 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 211549184 bytes 2021-02-20T13:11:47.064 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 223839232 bytes 2021-02-20T13:11:47.579 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 295615488 bytes 2021-02-20T13:11:47.882 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 350323712 bytes 2021-02-20T13:11:48.171 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 388773888 bytes 2021-02-20T13:11:48.627 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 444832768 bytes 2021-02-20T13:11:48.734 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 450439168 bytes 2021-02-20T13:11:48.925 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 470472704 bytes 2021-02-20T13:11:49.130 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 489194496 bytes 2021-02-20T13:11:49.475 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 532118528 bytes 2021-02-20T13:11:49.700 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 558598144 bytes 2021-02-20T13:11:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 721316864 bytes 2021-02-20T13:11:50.750 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 721906688 bytes 2021-02-20T13:11:51.000 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 775753728 bytes 2021-02-20T13:11:51.308 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 799640576 bytes 2021-02-20T13:11:51.512 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 840109056 bytes 2021-02-20T13:11:51.985 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 902863872 bytes 2021-02-20T13:11:52.096 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 905278464 bytes 2021-02-20T13:11:52.486 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 966667264 bytes 2021-02-20T13:11:52.710 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 993939456 bytes 2021-02-20T13:11:53.074 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 1044606976 bytes 2021-02-20T13:11:53.461 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 1102423040 bytes 2021-02-20T13:11:53.748 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 1138875392 bytes 2021-02-20T13:11:54.067 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2800 IOs, 1189690368 bytes 2021-02-20T13:11:54.256 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2900 IOs, 1207921664 bytes 2021-02-20T13:11:54.406 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3000 IOs, 1229907968 bytes 2021-02-20T13:11:54.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3100 IOs, 1243925504 bytes 2021-02-20T13:11:54.942 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3200 IOs, 1292369920 bytes 2021-02-20T13:11:55.270 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3300 IOs, 1335827456 bytes 2021-02-20T13:11:55.512 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3400 IOs, 1363021824 bytes 2021-02-20T13:11:55.763 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3500 IOs, 1396609024 bytes 2021-02-20T13:11:55.916 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3600 IOs, 1427805184 bytes 2021-02-20T13:11:56.177 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:11:56.178 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:11:56.178 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:11:56.178 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:11:56.179 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T13:11:56.179 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:11:56.179 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:56.179 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:56.179 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:11:56.180 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local description_pattern= 2021-02-20T13:11:56.180 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local service_pattern= 2021-02-20T13:11:56.180 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:11:57.179 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T13:11:57.179 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:11:57.180 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:57.180 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:57.180 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:11:57.180 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local description_pattern= 2021-02-20T13:11:57.181 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local service_pattern= 2021-02-20T13:11:57.181 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T13:11:57.181 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T13:11:57.181 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:11:57.182 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:11:57.182 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T13:11:57.183 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ tee /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:11:57.252 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:test: 2021-02-20T13:11:57.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: global_id: 49a0b5eb-7d69-4824-806b-1ece12354eb9 2021-02-20T13:11:57.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state: up+replaying 2021-02-20T13:11:57.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: description: replaying, {"bytes_per_second":15254903.47,"bytes_per_snapshot":486323712.0,"local_snapshot_timestamp":1613826691,"remote_snapshot_timestamp":1613826691,"replay_state":"idle"} 2021-02-20T13:11:57.254 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: service: mirror.0 on smithi183 2021-02-20T13:11:57.254 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: last_update: 2021-02-20 13:11:54 2021-02-20T13:11:57.254 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: peer_sites: 2021-02-20T13:11:57.254 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: name: cluster2 2021-02-20T13:11:57.255 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state: up+stopped 2021-02-20T13:11:57.255 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: description: local image is primary 2021-02-20T13:11:57.255 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: last_update: 2021-02-20 13:11:55 2021-02-20T13:11:57.262 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:11:57.264 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: state: up+replaying 2021-02-20T13:11:57.265 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^ description: .* /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:11:57.266 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: description: replaying, {"bytes_per_second":15254903.47,"bytes_per_snapshot":486323712.0,"local_snapshot_timestamp":1613826691,"remote_snapshot_timestamp":1613826691,"replay_state":"idle"} 2021-02-20T13:11:57.267 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:11:57.267 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo up+replaying 2021-02-20T13:11:57.267 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^up+ 2021-02-20T13:11:57.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:up+replaying 2021-02-20T13:11:57.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep service: *mirror..* on /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:11:57.270 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: service: mirror.0 on smithi183 2021-02-20T13:11:57.270 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:11:57.272 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local last_update=2021-02-20 13:11:54 2021-02-20T13:11:57.273 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 13:11:54 2021-02-20T13:11:57.273 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:11:57.273 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:57.273 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:57.274 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:11:57.274 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local prev_last_update=2021-02-20 13:11:54 2021-02-20T13:11:57.274 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T13:11:57.275 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T13:11:57.275 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:11:57.276 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror.mirror_status 2021-02-20T13:11:57.276 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T13:11:57.321 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T13:11:57.370 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local last_update state 2021-02-20T13:11:57.371 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T13:11:57.374 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ last_update=2021-02-20 13:11:54 2021-02-20T13:11:57.375 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T13:11:57.379 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ state=up+replaying 2021-02-20T13:11:57.380 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo up+replaying 2021-02-20T13:11:57.380 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep up+replaying 2021-02-20T13:11:57.381 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:up+replaying 2021-02-20T13:11:57.382 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:11:57.382 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:11:57.382 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap3 2021-02-20T13:11:57.383 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_snap cluster2 mirror test snap3 2021-02-20T13:11:57.383 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:11:57.383 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:11:57.384 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:11:57.384 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap3 2021-02-20T13:11:57.385 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap3 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:12:22.083 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4628520267036984) > weight (0.25) 2021-02-20T13:12:22.084 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:12:22.085 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:12:22.085 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T13:12:22.086 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T13:12:22.086 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:12:22.087 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T13:12:22.088+0000 7fca5b40e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 19187) UID: 1000 2021-02-20T13:12:22.286 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:12:26.131 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T13:12:26.132 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:12:26.132 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:12:26.132 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:12:26.133 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T13:12:26.133 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:12:26.134 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:12:26.134 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:12:26.134 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state=started 2021-02-20T13:12:26.135 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:12:26.135 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:12:27.133 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T13:12:27.134 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:12:27.134 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:12:27.134 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:12:27.134 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local test_state=started 2021-02-20T13:12:27.135 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_result 2021-02-20T13:12:27.135 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_state=stopped 2021-02-20T13:12:27.135 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T13:12:27.135 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ + local cluster_instance=cluster1grep 2021-02-20T13:12:27.135 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: -i state 2021-02-20T13:12:27.136 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:12:27.136 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:12:27.136 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance=cluster1 2021-02-20T13:12:27.136 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loop_instance 2021-02-20T13:12:27.137 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0 2021-02-20T13:12:27.138 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T13:12:27.138 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 0 3 2021-02-20T13:12:27.139 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T13:12:27.139 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:0 2021-02-20T13:12:27.139 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:12:27.139 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:12:27.140 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T13:12:27.140 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:12:27.140 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:12:27.140 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:12:27.140 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:12:27.141 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:12:27.141 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:12:27.141 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:12:27.141 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:12:27.142 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:12:27.142 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:12:27.142 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T13:12:27.142 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:0 2021-02-20T13:12:27.142 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:12:27.143 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:12:27.143 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T13:12:27.143 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:12:27.143 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:12:27.143 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:12:27.144 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:12:27.144 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:12:27.144 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:12:27.144 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:12:27.145 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:12:27.145 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:12:27.145 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:12:27.145 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T13:12:27.158 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:12:27.158 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:12:27.159 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:12:27.159 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T13:12:27.282 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:12:27.282 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ status_result= "state": "Replaying" 2021-02-20T13:12:27.282 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo "state": "Replaying" 2021-02-20T13:12:27.282 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep -i Replaying 2021-02-20T13:12:27.285 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: "state": "Replaying" 2021-02-20T13:12:27.285 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_state=started 2021-02-20T13:12:27.285 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test started = started 2021-02-20T13:12:27.285 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:12:27.286 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T13:12:27.286 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:12:27.286 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:12:27.286 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:12:27.288 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:12:27.288 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = journal ] 2021-02-20T13:12:27.288 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = snapshot ] 2021-02-20T13:12:27.289 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T13:12:27.289 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:12:27.289 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:12:27.289 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:12:27.290 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:12:27.290 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T13:12:27.290 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster2-mirror-test.status 2021-02-20T13:12:27.293 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:12:27.294 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:12:27.294 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T13:12:27.295 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror-test.status 2021-02-20T13:12:27.295 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:12:27.297 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:12:27.298 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T13:12:27.298 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:12:27.299 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:12:27.299 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:12:27.300 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T13:12:28.147 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Snapshot ID: 14 2021-02-20T13:12:28.161 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:12:28.161 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:12:28.161 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:12:28.162 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:12:28.162 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:12:28.162 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T13:12:28.162 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:12:28.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:12:28.190 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T13:12:28.190 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:12:28.190 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:12:28.193 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:12:28.193 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T13:12:28.218 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T13:12:28.221 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snapshot_id=14 2021-02-20T13:12:28.222 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:12:28.222 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.2 2021-02-20T13:12:28.423 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:12:28.423 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:12:28.423 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:12:28.424 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:12:28.424 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:12:28.424 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ + rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:12:28.424 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:12:28.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:12:28.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:12:28.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:12:28.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:12:28.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:12:28.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:12:28.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 14 = 2021-02-20T13:12:28.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.4 2021-02-20T13:12:28.895 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:12:28.895 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:12:28.895 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:12:28.896 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:12:28.896 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:12:28.896 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:12:28.896 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:12:28.965 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:12:28.966 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:12:28.969 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:12:28.970 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:12:28.970 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:12:28.970 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:12:28.970 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 14 = 2021-02-20T13:12:28.971 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.8 2021-02-20T13:12:29.772 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:12:29.772 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:12:29.772 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:12:29.772 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:12:29.773 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:12:29.773 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:12:29.773 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:12:29.841 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:12:29.842 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:12:29.846 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:12:29.847 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:12:29.847 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:12:29.847 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:12:29.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 14 = 2021-02-20T13:12:29.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1.6 2021-02-20T13:12:31.447 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:12:31.448 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:12:31.448 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:12:31.448 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:12:31.449 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:12:31.449 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:12:31.449 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:12:31.798 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:12:31.798 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:12:31.802 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:12:31.802 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:12:31.802 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:12:31.803 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:12:31.803 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 14 = 2021-02-20T13:12:31.803 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 2 2021-02-20T13:12:33.803 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:12:33.804 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:12:33.804 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:12:33.805 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:12:33.805 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:12:33.805 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:12:33.805 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:12:33.867 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:12:33.873 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id=14 2021-02-20T13:12:33.873 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 14 = 14 2021-02-20T13:12:33.873 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:12:33.874 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snap_present cluster1 mirror test snap3 2021-02-20T13:12:33.874 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:12:33.874 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:12:33.874 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:12:33.875 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap3 2021-02-20T13:12:33.875 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:12:33.875 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:12:34.874 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 info mirror/test@snap3 2021-02-20T13:12:34.935 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:rbd image 'test': 2021-02-20T13:12:34.936 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: size 512 MiB in 128 objects 2021-02-20T13:12:34.936 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: order 22 (4 MiB objects) 2021-02-20T13:12:34.937 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: snapshot_count: 6 2021-02-20T13:12:34.937 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: id: 109d36d23f3 2021-02-20T13:12:34.938 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: block_name_prefix: rbd_data.109d36d23f3 2021-02-20T13:12:34.938 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: format: 2 2021-02-20T13:12:34.939 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: features: layering, exclusive-lock, non-primary 2021-02-20T13:12:34.939 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: op_features: 2021-02-20T13:12:34.940 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: flags: 2021-02-20T13:12:34.940 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: create_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:12:34.940 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: access_timestamp: Sat Feb 20 13:11:40 2021 2021-02-20T13:12:34.941 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: modify_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:12:34.941 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: protected: False 2021-02-20T13:12:34.942 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring state: enabled 2021-02-20T13:12:34.942 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring mode: snapshot 2021-02-20T13:12:34.943 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring global id: 49a0b5eb-7d69-4824-806b-1ece12354eb9 2021-02-20T13:12:34.943 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring primary: false 2021-02-20T13:12:34.945 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:12:34.945 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snap2-clean ] 2021-02-20T13:12:34.946 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap2-clean 2021-02-20T13:12:34.946 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:12:34.946 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:12:34.946 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap2-clean 2021-02-20T13:12:34.947 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:12:34.947 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:12:34.947 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:12:34.947 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:12:34.947 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2-clean /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:12:35.757 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:12:35.760 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2-clean /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:12:36.755 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:12:36.759 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:12:37.043 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:12:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:12:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap3 2021-02-20T13:12:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:12:37.173 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:12:37.173 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap3 2021-02-20T13:12:37.173 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:12:37.173 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:12:37.173 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:12:37.174 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:12:37.174 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3 /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:12:37.984 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:12:37.990 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3 /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:12:38.113 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:12:38.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:12:38.119 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T13:12:38.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T13:12:38.120+0000 7fab74e7d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 19107) UID: 1000 2021-02-20T13:12:38.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:12:39.007 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:12:39.011 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:12:39.298 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:12:39.424 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:12:39.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ clean_snap_name=snap3-clean 2021-02-20T13:12:39.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_snap cluster2 mirror test snap3-clean 2021-02-20T13:12:39.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:12:39.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:12:39.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:12:39.426 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap3-clean 2021-02-20T13:12:39.426 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap3-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T13:12:40.106 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T13:12:40.107 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:12:40.107 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:12:40.107 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:12:40.107 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:12:40.108 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=6 2021-02-20T13:12:40.108 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:12:40.108 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T13:12:41.234 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 65186816 bytes 2021-02-20T13:12:41.926 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 97458176 bytes 2021-02-20T13:12:42.120 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:12:42.122 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:12:42.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:12:42.122 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:12:42.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:12:42.123 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:12:42.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:12:42.126 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T13:12:42.693 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 180654080 bytes 2021-02-20T13:12:43.139 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 225050624 bytes 2021-02-20T13:12:43.689 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 285288448 bytes 2021-02-20T13:12:43.866 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 311472128 bytes 2021-02-20T13:12:44.663 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 359928832 bytes 2021-02-20T13:12:44.950 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 388533248 bytes 2021-02-20T13:12:45.361 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 438765568 bytes 2021-02-20T13:12:45.612 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 470252544 bytes 2021-02-20T13:12:46.060 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 505418752 bytes 2021-02-20T13:12:46.112 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:12:46.112 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:12:46.112 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:12:46.112 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:12:46.113 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T13:12:46.113 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:12:46.113 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:12:46.113 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:12:46.114 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:12:46.114 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local description_pattern= 2021-02-20T13:12:46.114 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local service_pattern= 2021-02-20T13:12:46.114 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:12:47.114 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T13:12:47.114 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:12:47.115 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:12:47.115 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:12:47.115 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:12:47.115 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local description_pattern= 2021-02-20T13:12:47.116 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local service_pattern= 2021-02-20T13:12:47.116 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T13:12:47.116 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T13:12:47.116 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:12:47.117 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:12:47.117 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T13:12:47.118 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ tee /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:12:47.186 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:test: 2021-02-20T13:12:47.187 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: global_id: 49a0b5eb-7d69-4824-806b-1ece12354eb9 2021-02-20T13:12:47.187 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state: up+replaying 2021-02-20T13:12:47.187 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"local_snapshot_timestamp":1613826748,"remote_snapshot_timestamp":1613826748,"replay_state":"idle"} 2021-02-20T13:12:47.188 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: service: mirror.3 on smithi183 2021-02-20T13:12:47.188 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: last_update: 2021-02-20 13:12:42 2021-02-20T13:12:47.188 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: peer_sites: 2021-02-20T13:12:47.189 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: name: cluster2 2021-02-20T13:12:47.189 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state: up+stopped 2021-02-20T13:12:47.189 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: description: local image is primary 2021-02-20T13:12:47.189 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: last_update: 2021-02-20 13:12:25 2021-02-20T13:12:47.196 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:12:47.198 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: state: up+replaying 2021-02-20T13:12:47.199 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^ description: .* /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:12:47.200 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"local_snapshot_timestamp":1613826748,"remote_snapshot_timestamp":1613826748,"replay_state":"idle"} 2021-02-20T13:12:47.201 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:up+replaying 2021-02-20T13:12:47.201 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:12:47.201 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^up+ 2021-02-20T13:12:47.202 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo up+replaying 2021-02-20T13:12:47.203 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep service: *mirror..* on /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:12:47.206 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: service: mirror.3 on smithi183 2021-02-20T13:12:47.207 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:12:47.207 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local last_update=2021-02-20 13:12:42 2021-02-20T13:12:47.207 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 13:12:42 2021-02-20T13:12:47.208 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:12:47.208 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:12:47.208 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:12:47.208 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:12:47.208 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local prev_last_update=2021-02-20 13:12:42 2021-02-20T13:12:47.209 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T13:12:47.209 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T13:12:47.210 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:12:47.210 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror.mirror_status 2021-02-20T13:12:47.210 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T13:12:47.250 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T13:12:47.297 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local last_update state 2021-02-20T13:12:47.297 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T13:12:47.302 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ last_update=2021-02-20 13:12:42 2021-02-20T13:12:47.303 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T13:12:47.311 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ state=up+replaying 2021-02-20T13:12:47.311 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo up+replaying 2021-02-20T13:12:47.311 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep up+replaying 2021-02-20T13:12:47.311 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:12:47.312 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:12:47.312 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap4 2021-02-20T13:12:47.312 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_snap cluster2 mirror test snap4 2021-02-20T13:12:47.312 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:12:47.313 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:12:47.313 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:12:47.313 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap4 2021-02-20T13:12:47.314 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap4 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:12:47.314 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:up+replaying 2021-02-20T13:13:17.130 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T13:13:17.130 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:13:17.130 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:13:17.131 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:13:17.131 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T13:13:17.131 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:13:17.131 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:13:17.132 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:13:17.132 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state=started 2021-02-20T13:13:17.132 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:13:17.132 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:13:17.133 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T13:13:17.136+0000 7f03ccd38700 -1 reset not still connected to 0x556b850c0150 2021-02-20T13:13:18.131 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T13:13:18.131 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:13:18.131 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:13:18.132 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:13:18.132 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local test_state=started 2021-02-20T13:13:18.132 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_result 2021-02-20T13:13:18.132 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_state=stopped 2021-02-20T13:13:18.133 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T13:13:18.134 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_instance=cluster1 2021-02-20T13:13:18.134 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ + shift 2021-02-20T13:13:18.134 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:grep -i state+ 2021-02-20T13:13:18.134 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:local cluster=cluster1 2021-02-20T13:13:18.135 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance=cluster1 2021-02-20T13:13:18.135 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loop_instance 2021-02-20T13:13:18.135 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0 2021-02-20T13:13:18.135 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T13:13:18.135 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 0 3 2021-02-20T13:13:18.137 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T13:13:18.138 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:0 2021-02-20T13:13:18.138 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:13:18.138 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:13:18.139 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T13:13:18.139 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:13:18.140 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:13:18.140 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:13:18.140 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:13:18.141 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:13:18.141 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:13:18.141 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:13:18.141 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:13:18.142 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:13:18.142 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:13:18.142 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T13:13:18.142 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:0 2021-02-20T13:13:18.143 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:13:18.143 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:13:18.143 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T13:13:18.143 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:13:18.144 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:13:18.144 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:13:18.144 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:13:18.144 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:13:18.145 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:13:18.145 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:13:18.145 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:13:18.145 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:13:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:13:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T13:13:18.157 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:13:18.158 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:13:18.158 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:13:18.158 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T13:13:18.265 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:13:18.265 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:13:18.266 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:13:18.267 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T13:13:18.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:1 2021-02-20T13:13:18.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:13:18.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:13:18.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T13:13:18.269 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:13:18.269 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:13:18.269 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:13:18.269 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:13:18.269 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:13:18.270 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:13:18.270 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:13:18.270 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:13:18.270 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:13:18.271 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:13:18.271 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T13:13:18.271 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:1 2021-02-20T13:13:18.271 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:13:18.272 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:13:18.272 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T13:13:18.272 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:13:18.272 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:13:18.272 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:13:18.273 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:13:18.273 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:13:18.273 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:13:18.273 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:13:18.273 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:13:18.274 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:13:18.274 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:13:18.274 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T13:13:18.286 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:13:18.287 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:13:18.287 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:13:18.287 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T13:13:18.386 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:13:18.387 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:13:18.387 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:13:18.389 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T13:13:18.389 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:2 2021-02-20T13:13:18.390 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:13:18.390 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:13:18.390 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T13:13:18.390 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:13:18.390 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:13:18.391 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:13:18.391 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:13:18.391 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:13:18.391 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:13:18.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:13:18.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:13:18.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:13:18.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:13:18.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T13:13:18.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:2 2021-02-20T13:13:18.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:13:18.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:13:18.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T13:13:18.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:13:18.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:13:18.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:13:18.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:13:18.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:13:18.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:13:18.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:13:18.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:13:18.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:13:18.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:13:18.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T13:13:18.411 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:13:18.412 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:13:18.412 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:13:18.412 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T13:13:18.510 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:13:18.510 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:13:18.510 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:13:18.512 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T13:13:18.512 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:3 2021-02-20T13:13:18.512 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:13:18.512 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:13:18.513 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T13:13:18.513 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:13:18.513 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:13:18.513 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:13:18.514 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:13:18.514 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:13:18.514 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:13:18.514 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:13:18.515 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:13:18.515 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:13:18.515 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:13:18.515 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T13:13:18.515 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:3 2021-02-20T13:13:18.516 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:13:18.516 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:13:18.516 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T13:13:18.516 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:13:18.516 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:13:18.517 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:13:18.517 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:13:18.517 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:13:18.517 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:13:18.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:13:18.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:13:18.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:13:18.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:13:18.519 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T13:13:18.531 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:13:18.531 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:13:18.531 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:13:18.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T13:13:18.649 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:13:18.650 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ status_result= "state": "Replaying" 2021-02-20T13:13:18.651 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo "state": "Replaying" 2021-02-20T13:13:18.651 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep -i Replaying 2021-02-20T13:13:18.652 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: "state": "Replaying" 2021-02-20T13:13:18.652 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_state=started 2021-02-20T13:13:18.653 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test started = started 2021-02-20T13:13:18.653 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:13:18.653 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T13:13:18.653 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:13:18.654 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:13:18.654 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:13:18.654 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:13:18.654 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = journal ] 2021-02-20T13:13:18.654 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = snapshot ] 2021-02-20T13:13:18.655 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T13:13:18.655 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:13:18.656 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:13:18.656 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:13:18.656 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:13:18.656 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T13:13:18.656 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster2-mirror-test.status 2021-02-20T13:13:18.657 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:13:18.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:13:18.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T13:13:18.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror-test.status 2021-02-20T13:13:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:13:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:13:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T13:13:18.660 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:13:18.660 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:13:18.660 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:13:18.661 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T13:13:19.138 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Snapshot ID: 18 2021-02-20T13:13:19.152 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:13:19.152 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:13:19.152 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:13:19.152 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:13:19.153 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:13:19.153 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T13:13:19.153 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:13:19.214 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T13:13:19.218 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snapshot_id=18 2021-02-20T13:13:19.218 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:13:19.218 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.2 2021-02-20T13:13:19.422 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:13:19.423 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:13:19.423 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:13:19.423 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:13:19.424 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:13:19.424 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:13:19.424 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:13:19.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:13:19.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id=14 2021-02-20T13:13:19.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 18 = 14 2021-02-20T13:13:19.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.4 2021-02-20T13:13:19.895 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:13:19.895 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:13:19.895 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:13:19.895 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:13:19.896 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:13:19.896 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:13:19.896 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:13:19.967 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:13:19.967 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:13:19.971 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:13:19.971 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:13:19.972 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:13:19.972 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:13:19.972 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 18 = 2021-02-20T13:13:19.972 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.8 2021-02-20T13:13:20.774 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:13:20.775 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:13:20.775 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:13:20.775 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:13:20.775 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:13:20.775 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:13:20.776 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:13:20.840 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:13:20.841 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:13:20.844 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:13:20.844 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:13:20.845 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:13:20.845 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:13:20.846 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 18 = 2021-02-20T13:13:20.846 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1.6 2021-02-20T13:13:22.447 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:13:22.447 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:13:22.448 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:13:22.448 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:13:22.449 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:13:22.449 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:13:22.449 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:13:22.512 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:13:22.513 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:13:22.516 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:13:22.516 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:13:22.517 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:13:22.517 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:13:22.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 18 = 2021-02-20T13:13:22.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 2 2021-02-20T13:13:24.517 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:13:24.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:13:24.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:13:24.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:13:24.519 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:13:24.519 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:13:24.519 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:13:24.582 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:13:24.587 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id=18 2021-02-20T13:13:24.588 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 18 = 18 2021-02-20T13:13:24.588 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:13:24.588 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snap_present cluster1 mirror test snap4 2021-02-20T13:13:24.588 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:13:24.588 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:13:24.589 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:13:24.589 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap4 2021-02-20T13:13:24.589 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:13:24.589 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:13:25.589 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 info mirror/test@snap4 2021-02-20T13:13:25.657 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:rbd image 'test': 2021-02-20T13:13:25.657 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: size 512 MiB in 128 objects 2021-02-20T13:13:25.657 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: order 22 (4 MiB objects) 2021-02-20T13:13:25.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: snapshot_count: 9 2021-02-20T13:13:25.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: id: 109d36d23f3 2021-02-20T13:13:25.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: block_name_prefix: rbd_data.109d36d23f3 2021-02-20T13:13:25.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: format: 2 2021-02-20T13:13:25.659 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: features: layering, exclusive-lock, operations, non-primary 2021-02-20T13:13:25.659 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: op_features: snap-trash 2021-02-20T13:13:25.659 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: flags: 2021-02-20T13:13:25.659 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: create_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:13:25.659 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: access_timestamp: Sat Feb 20 13:11:40 2021 2021-02-20T13:13:25.660 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: modify_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:13:25.660 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: protected: False 2021-02-20T13:13:25.660 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring state: enabled 2021-02-20T13:13:25.660 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring mode: snapshot 2021-02-20T13:13:25.660 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring global id: 49a0b5eb-7d69-4824-806b-1ece12354eb9 2021-02-20T13:13:25.661 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring primary: false 2021-02-20T13:13:25.663 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:13:25.663 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snap3-clean ] 2021-02-20T13:13:25.663 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap3-clean 2021-02-20T13:13:25.664 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:13:25.664 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:13:25.664 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap3-clean 2021-02-20T13:13:25.664 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:13:25.664 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:13:25.665 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:13:25.665 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:13:25.665 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3-clean /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:13:26.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:13:26.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3-clean /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:13:27.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:13:27.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:13:27.781 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:13:27.907 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:13:27.908 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap4 2021-02-20T13:13:27.908 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:13:27.908 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:13:27.908 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap4 2021-02-20T13:13:27.909 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:13:27.909 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:13:27.909 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:13:27.909 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:13:27.910 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4 /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:13:28.195 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.575698752001817) > weight (0.25) 2021-02-20T13:13:28.195 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:13:28.196 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:13:28.196 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T13:13:28.197 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T13:13:28.200+0000 7f891bc00700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 23188) UID: 1000 2021-02-20T13:13:28.397 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:13:28.732 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:13:28.736 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4 /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:13:29.786 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:13:29.790 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:13:30.069 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:13:30.197 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:13:30.197 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ clean_snap_name=snap4-clean 2021-02-20T13:13:30.197 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_snap cluster2 mirror test snap4-clean 2021-02-20T13:13:30.197 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:13:30.198 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:13:30.198 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:13:30.198 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap4-clean 2021-02-20T13:13:30.198 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap4-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T13:13:31.153 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T13:13:31.153 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:13:31.154 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:13:31.154 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:13:31.155 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:13:31.156 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=7 2021-02-20T13:13:31.156 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:13:31.156 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T13:13:32.197 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 47542272 bytes 2021-02-20T13:13:32.730 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 71823360 bytes 2021-02-20T13:13:32.971 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 76988416 bytes 2021-02-20T13:13:33.283 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 110201856 bytes 2021-02-20T13:13:33.446 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 110813184 bytes 2021-02-20T13:13:33.747 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 150761472 bytes 2021-02-20T13:13:34.027 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 171336704 bytes 2021-02-20T13:13:34.197 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T13:13:34.198 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:13:34.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:13:34.199 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T13:13:34.199 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:13:34.200 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:13:34.203 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:13:34.205 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T13:13:34.245 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 200118272 bytes 2021-02-20T13:13:34.504 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 235730944 bytes 2021-02-20T13:13:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 309258240 bytes 2021-02-20T13:13:35.387 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 325145600 bytes 2021-02-20T13:13:35.975 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 398142464 bytes 2021-02-20T13:13:36.292 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 429586432 bytes 2021-02-20T13:13:36.592 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 467367936 bytes 2021-02-20T13:13:36.859 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 509508608 bytes 2021-02-20T13:13:37.126 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 526605312 bytes 2021-02-20T13:13:37.576 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 583204864 bytes 2021-02-20T13:13:37.785 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 599554048 bytes 2021-02-20T13:13:38.160 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:13:38.160 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:13:38.160 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:13:38.161 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:13:38.161 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T13:13:38.161 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:13:38.161 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:13:38.162 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:13:38.162 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:13:38.162 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local description_pattern= 2021-02-20T13:13:38.162 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local service_pattern= 2021-02-20T13:13:38.162 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:13:39.160 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T13:13:39.160 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:13:39.160 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:13:39.161 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:13:39.161 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:13:39.161 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local description_pattern= 2021-02-20T13:13:39.161 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local service_pattern= 2021-02-20T13:13:39.162 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T13:13:39.162 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T13:13:39.162 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:13:39.163 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:13:39.163 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T13:13:39.164 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ tee /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:13:39.233 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:test: 2021-02-20T13:13:39.234 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: global_id: 49a0b5eb-7d69-4824-806b-1ece12354eb9 2021-02-20T13:13:39.234 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state: up+replaying 2021-02-20T13:13:39.234 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: description: replaying, {"bytes_per_second":1386973.87,"bytes_per_snapshot":0.0,"local_snapshot_timestamp":1613826748,"remote_snapshot_timestamp":1613826799,"replay_state":"syncing","seconds_until_synced":0,"syncing_percent":11,"syncing_snapshot_timestamp":1613826799} 2021-02-20T13:13:39.234 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: service: mirror.3 on smithi183 2021-02-20T13:13:39.234 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: last_update: 2021-02-20 13:13:25 2021-02-20T13:13:39.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: peer_sites: 2021-02-20T13:13:39.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: name: cluster2 2021-02-20T13:13:39.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state: up+stopped 2021-02-20T13:13:39.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: description: local image is primary 2021-02-20T13:13:39.236 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: last_update: 2021-02-20 13:13:25 2021-02-20T13:13:39.244 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:13:39.245 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: state: up+replaying 2021-02-20T13:13:39.245 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^ description: .* /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:13:39.247 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: description: replaying, {"bytes_per_second":1386973.87,"bytes_per_snapshot":0.0,"local_snapshot_timestamp":1613826748,"remote_snapshot_timestamp":1613826799,"replay_state":"syncing","seconds_until_synced":0,"syncing_percent":11,"syncing_snapshot_timestamp":1613826799} 2021-02-20T13:13:39.247 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:13:39.247 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo up+replaying 2021-02-20T13:13:39.249 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^up+ 2021-02-20T13:13:39.250 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:up+replaying 2021-02-20T13:13:39.250 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: service: mirror.3 on smithi183 2021-02-20T13:13:39.250 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep service: *mirror..* on /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:13:39.251 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:13:39.252 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local last_update=2021-02-20 13:13:25 2021-02-20T13:13:39.252 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 13:13:25 2021-02-20T13:13:39.252 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:13:39.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:13:39.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:13:39.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:13:39.254 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local prev_last_update=2021-02-20 13:13:25 2021-02-20T13:13:39.254 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T13:13:39.254 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T13:13:39.254 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:13:39.255 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror.mirror_status 2021-02-20T13:13:39.255 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T13:13:39.298 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T13:13:39.350 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local last_update state 2021-02-20T13:13:39.351 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T13:13:39.356 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ last_update=2021-02-20 13:13:25 2021-02-20T13:13:39.357 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T13:13:39.362 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ state=up+replaying 2021-02-20T13:13:39.362 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo up+replaying 2021-02-20T13:13:39.363 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep up+replaying 2021-02-20T13:13:39.366 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:up+replaying 2021-02-20T13:13:39.366 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:13:39.367 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:13:39.367 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap5 2021-02-20T13:13:39.367 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_snap cluster2 mirror test snap5 2021-02-20T13:13:39.368 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:13:39.368 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:13:39.369 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:13:39.369 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap5 2021-02-20T13:13:39.370 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap5 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:14:09.222 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T13:14:09.223 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:14:09.223 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:09.223 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:09.224 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T13:14:09.224 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:14:09.224 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:09.224 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:09.225 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state=started 2021-02-20T13:14:09.225 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:14:09.225 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:14:10.225 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T13:14:10.226 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:14:10.226 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:10.227 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:10.227 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local test_state=started 2021-02-20T13:14:10.227 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_result 2021-02-20T13:14:10.228 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_state=stopped 2021-02-20T13:14:10.228 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test+ 2021-02-20T13:14:10.228 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:grep+ -i statelocal 2021-02-20T13:14:10.236 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: cluster_instance=cluster1 2021-02-20T13:14:10.236 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:14:10.236 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:14:10.236 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance=cluster1 2021-02-20T13:14:10.237 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loop_instance 2021-02-20T13:14:10.237 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0 2021-02-20T13:14:10.237 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T13:14:10.241 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 0 3 2021-02-20T13:14:10.241 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T13:14:10.242 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:0 2021-02-20T13:14:10.242 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:14:10.242 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:14:10.242 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T13:14:10.243 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:14:10.243 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:14:10.243 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:14:10.243 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:14:10.244 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:14:10.244 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:14:10.244 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:14:10.244 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:14:10.245 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:14:10.245 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:14:10.245 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T13:14:10.246 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:0 2021-02-20T13:14:10.246 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:14:10.246 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:14:10.246 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T13:14:10.247 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:14:10.247 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:14:10.247 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:14:10.247 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:14:10.248 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:14:10.249 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:14:10.249 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:14:10.249 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:14:10.250 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:14:10.250 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:14:10.250 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T13:14:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:14:10.259 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:14:10.259 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:14:10.259 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T13:14:10.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:14:10.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:14:10.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:14:10.355 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T13:14:10.356 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:1 2021-02-20T13:14:10.358 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:14:10.359 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:14:10.359 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T13:14:10.359 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:14:10.360 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:14:10.360 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:14:10.360 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:14:10.360 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:14:10.360 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:14:10.361 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:14:10.361 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:14:10.361 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:14:10.361 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:14:10.362 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T13:14:10.362 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:1 2021-02-20T13:14:10.362 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:14:10.362 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:14:10.362 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T13:14:10.363 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:14:10.363 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:14:10.363 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:14:10.363 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:14:10.364 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:14:10.364 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:14:10.364 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:14:10.364 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:14:10.365 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:14:10.365 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:14:10.365 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T13:14:10.373 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:14:10.373 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:14:10.373 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:14:10.374 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T13:14:10.471 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:14:10.472 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:14:10.472 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:14:10.476 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T13:14:10.476 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:2 2021-02-20T13:14:10.477 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:14:10.477 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:14:10.478 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T13:14:10.478 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:14:10.478 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:14:10.479 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:14:10.479 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:14:10.479 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:14:10.479 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:14:10.480 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:14:10.480 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:14:10.480 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:14:10.480 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:14:10.481 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T13:14:10.481 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:2 2021-02-20T13:14:10.481 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:14:10.481 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:14:10.482 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T13:14:10.482 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:14:10.482 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:14:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:14:10.484 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:14:10.484 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:14:10.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:14:10.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:14:10.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:14:10.486 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:14:10.486 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:14:10.486 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T13:14:10.496 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:14:10.496 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:14:10.496 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:14:10.497 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T13:14:10.592 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:14:10.592 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:14:10.592 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:14:10.595 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T13:14:10.595 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:3 2021-02-20T13:14:10.596 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:14:10.596 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:14:10.596 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T13:14:10.597 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:14:10.597 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:14:10.597 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:14:10.597 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:14:10.597 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:14:10.598 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:14:10.598 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:14:10.598 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:14:10.598 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:14:10.599 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:14:10.599 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T13:14:10.599 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:3 2021-02-20T13:14:10.599 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:14:10.600 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:14:10.600 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T13:14:10.600 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:14:10.600 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:14:10.600 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:14:10.601 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:14:10.601 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:14:10.601 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:14:10.602 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:14:10.602 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:14:10.602 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:14:10.602 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:14:10.603 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T13:14:10.615 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:14:10.615 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:14:10.616 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:14:10.616 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T13:14:10.724 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:14:10.725 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ status_result= "state": "Replaying" 2021-02-20T13:14:10.726 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo "state": "Replaying" 2021-02-20T13:14:10.726 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep -i Replaying 2021-02-20T13:14:10.727 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: "state": "Replaying" 2021-02-20T13:14:10.729 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_state=started 2021-02-20T13:14:10.729 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test started = started 2021-02-20T13:14:10.729 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:14:10.730 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T13:14:10.730 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:14:10.730 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:14:10.730 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:10.730 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:10.731 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = journal ] 2021-02-20T13:14:10.731 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = snapshot ] 2021-02-20T13:14:10.731 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T13:14:10.731 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:14:10.732 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:14:10.732 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:10.732 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:10.732 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T13:14:10.732 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster2-mirror-test.status 2021-02-20T13:14:10.733 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:14:10.734 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:14:10.734 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T13:14:10.734 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror-test.status 2021-02-20T13:14:10.735 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:14:10.735 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:14:10.735 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T13:14:10.735 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:14:10.735 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:10.736 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:10.736 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T13:14:11.236 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Snapshot ID: 22 2021-02-20T13:14:11.444 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:14:11.444 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:14:11.445 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:11.445 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:11.445 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:14:11.445 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T13:14:11.446 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:14:11.450 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T13:14:11.451 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snapshot_id=22 2021-02-20T13:14:11.451 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:14:11.451 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.2 2021-02-20T13:14:11.517 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:14:11.517 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:14:11.517 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:14:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:14:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:14:11.588 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:14:11.594 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id=18 2021-02-20T13:14:11.594 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 22 = 18 2021-02-20T13:14:11.594 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.4 2021-02-20T13:14:11.995 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:14:11.996 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:14:11.996 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:11.996 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:11.997 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:14:11.997 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:14:11.997 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:14:12.065 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:14:12.066 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:14:12.076 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:14:12.076 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:14:12.076 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:14:12.077 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:14:12.077 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 22 = 2021-02-20T13:14:12.077 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.8 2021-02-20T13:14:12.872 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:14:12.873 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:14:12.873 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:12.873 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:12.873 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:14:12.874 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:14:12.874 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:14:12.940 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:14:12.940 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:14:12.943 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:14:12.943 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:14:12.944 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:14:12.944 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:14:12.945 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 22 = 2021-02-20T13:14:12.945 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1.6 2021-02-20T13:14:14.546 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:14:14.546 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:14:14.546 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:14.547 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:14.547 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:14:14.547 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:14:14.547 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:14:14.617 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:14:14.618 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:14:14.623 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:14:14.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:14:14.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:14:14.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:14:14.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 22 = 2021-02-20T13:14:14.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 2 2021-02-20T13:14:16.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:14:16.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:14:16.625 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:16.625 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:16.625 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:14:16.625 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:14:16.627 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:14:16.731 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:14:16.732 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:14:16.737 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:14:16.737 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:14:16.737 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:14:16.738 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:14:16.738 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 22 = 2021-02-20T13:14:16.738 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 2 2021-02-20T13:14:18.739 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:14:18.739 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:14:18.739 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:18.740 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:18.740 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:14:18.740 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:14:18.740 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:14:18.810 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:14:18.814 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id=22 2021-02-20T13:14:18.815 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 22 = 22 2021-02-20T13:14:18.815 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:14:18.815 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snap_present cluster1 mirror test snap5 2021-02-20T13:14:18.815 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:14:18.816 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:18.816 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:18.816 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap5 2021-02-20T13:14:18.816 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:14:18.817 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:14:19.816 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 info mirror/test@snap5 2021-02-20T13:14:19.882 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:rbd image 'test': 2021-02-20T13:14:19.882 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: size 512 MiB in 128 objects 2021-02-20T13:14:19.882 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: order 22 (4 MiB objects) 2021-02-20T13:14:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: snapshot_count: 10 2021-02-20T13:14:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: id: 109d36d23f3 2021-02-20T13:14:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: block_name_prefix: rbd_data.109d36d23f3 2021-02-20T13:14:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: format: 2 2021-02-20T13:14:19.884 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: features: layering, exclusive-lock, non-primary 2021-02-20T13:14:19.884 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: op_features: 2021-02-20T13:14:19.884 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: flags: 2021-02-20T13:14:19.884 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: create_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:14:19.884 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: access_timestamp: Sat Feb 20 13:13:26 2021 2021-02-20T13:14:19.885 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: modify_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:14:19.885 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: protected: False 2021-02-20T13:14:19.885 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring state: enabled 2021-02-20T13:14:19.885 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring mode: snapshot 2021-02-20T13:14:19.886 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring global id: 49a0b5eb-7d69-4824-806b-1ece12354eb9 2021-02-20T13:14:19.886 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring primary: false 2021-02-20T13:14:19.887 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:14:19.887 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snap4-clean ] 2021-02-20T13:14:19.887 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap4-clean 2021-02-20T13:14:19.888 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:19.888 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:19.888 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap4-clean 2021-02-20T13:14:19.889 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:14:19.889 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:14:19.889 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:14:19.889 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:14:19.890 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4-clean /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:14:20.710 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:14:20.712 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4-clean /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:14:21.754 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:14:21.759 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:14:22.037 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:14:22.165 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:14:22.165 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap5 2021-02-20T13:14:22.165 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:22.165 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:22.166 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap5 2021-02-20T13:14:22.166 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:14:22.166 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:14:22.166 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:14:22.167 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:14:22.167 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5 /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:14:22.504 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T13:14:22.504+0000 7fcbc3699700 -1 bdev(0x55c6925e8000 /var/lib/ceph/osd/cluster2-2/block) aio_submit retries 1 2021-02-20T13:14:22.966 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:14:22.970 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5 /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:14:23.884 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:14:23.888 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:14:24.165 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:14:24.292 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:14:24.292 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ clean_snap_name=snap5-clean 2021-02-20T13:14:24.293 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_snap cluster2 mirror test snap5-clean 2021-02-20T13:14:24.293 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:14:24.293 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:24.293 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:24.294 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap5-clean 2021-02-20T13:14:24.294 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap5-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T13:14:25.138 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T13:14:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:14:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:25.140 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:25.140 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:14:25.142 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=9 2021-02-20T13:14:25.143 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:14:25.143 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T13:14:25.969 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 1072128 bytes 2021-02-20T13:14:26.683 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 48047104 bytes 2021-02-20T13:14:27.201 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 97591296 bytes 2021-02-20T13:14:27.651 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 140666880 bytes 2021-02-20T13:14:27.942 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 170913792 bytes 2021-02-20T13:14:28.300 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 206843904 bytes 2021-02-20T13:14:28.755 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 256942080 bytes 2021-02-20T13:14:29.082 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 304531456 bytes 2021-02-20T13:14:29.617 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 356113408 bytes 2021-02-20T13:14:29.845 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 376570880 bytes 2021-02-20T13:14:30.036 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 388418560 bytes 2021-02-20T13:14:30.333 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 443443200 bytes 2021-02-20T13:14:30.590 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 451218432 bytes 2021-02-20T13:14:30.960 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 505768960 bytes 2021-02-20T13:14:31.127 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 506354688 bytes 2021-02-20T13:14:31.129 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2603576999791559) > weight (0.25) 2021-02-20T13:14:31.129 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.74923687141447) > weight (0.25) 2021-02-20T13:14:31.129 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T13:14:31.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T13:14:31.130 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T13:14:31.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T13:14:31.136+0000 7fb87f768700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 20493) UID: 1000 2021-02-20T13:14:31.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:14:31.456 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 538080256 bytes 2021-02-20T13:14:31.584 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 538697728 bytes 2021-02-20T13:14:31.987 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 598983680 bytes 2021-02-20T13:14:32.220 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 621700096 bytes 2021-02-20T13:14:32.702 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 692142080 bytes 2021-02-20T13:14:32.975 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 712164352 bytes 2021-02-20T13:14:33.168 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 725125120 bytes 2021-02-20T13:14:33.517 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 768861184 bytes 2021-02-20T13:14:33.825 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 803397632 bytes 2021-02-20T13:14:34.145 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:14:34.145 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:14:34.145 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:14:34.145 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:14:34.146 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T13:14:34.146 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:14:34.146 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:34.146 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:34.147 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:14:34.147 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local description_pattern= 2021-02-20T13:14:34.147 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local service_pattern= 2021-02-20T13:14:34.147 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:14:35.130 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T13:14:35.131 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:14:35.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:14:35.132 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T13:14:35.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T13:14:35.132 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T13:14:35.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T13:14:35.136 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T13:14:35.146 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T13:14:35.146 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:14:35.146 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:35.147 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:35.147 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:14:35.147 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local description_pattern= 2021-02-20T13:14:35.147 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local service_pattern= 2021-02-20T13:14:35.147 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T13:14:35.149 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T13:14:35.149 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:14:35.149 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:14:35.150 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T13:14:35.150 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ tee /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:14:35.219 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:test: 2021-02-20T13:14:35.219 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: global_id: 49a0b5eb-7d69-4824-806b-1ece12354eb9 2021-02-20T13:14:35.219 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state: up+replaying 2021-02-20T13:14:35.219 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: description: replaying, {"bytes_per_second":12577587.2,"bytes_per_snapshot":342288896.0,"local_snapshot_timestamp":1613826851,"remote_snapshot_timestamp":1613826851,"replay_state":"idle"} 2021-02-20T13:14:35.220 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: service: mirror.3 on smithi183 2021-02-20T13:14:35.220 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: last_update: 2021-02-20 13:14:25 2021-02-20T13:14:35.220 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: peer_sites: 2021-02-20T13:14:35.220 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: name: cluster2 2021-02-20T13:14:35.220 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state: up+stopped 2021-02-20T13:14:35.221 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: description: local image is primary 2021-02-20T13:14:35.221 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: last_update: 2021-02-20 13:14:25 2021-02-20T13:14:35.228 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:14:35.230 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: state: up+replaying 2021-02-20T13:14:35.230 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^ description: .* /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:14:35.231 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: description: replaying, {"bytes_per_second":12577587.2,"bytes_per_snapshot":342288896.0,"local_snapshot_timestamp":1613826851,"remote_snapshot_timestamp":1613826851,"replay_state":"idle"} 2021-02-20T13:14:35.232 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:14:35.232 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo up+replaying 2021-02-20T13:14:35.232 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^up+ 2021-02-20T13:14:35.233 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:up+replaying 2021-02-20T13:14:35.233 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep service: *mirror..* on /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:14:35.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: service: mirror.3 on smithi183 2021-02-20T13:14:35.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:14:35.237 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local last_update=2021-02-20 13:14:25 2021-02-20T13:14:35.237 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 13:14:25 2021-02-20T13:14:35.238 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:14:35.238 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:35.238 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:35.238 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:14:35.238 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local prev_last_update=2021-02-20 13:14:25 2021-02-20T13:14:35.239 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T13:14:35.240 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T13:14:35.240 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:14:35.240 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror.mirror_status 2021-02-20T13:14:35.241 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T13:14:35.289 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T13:14:35.332 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local last_update state 2021-02-20T13:14:35.333 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T13:14:35.335 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ last_update=2021-02-20 13:14:25 2021-02-20T13:14:35.336 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T13:14:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ state=up+replaying 2021-02-20T13:14:35.339 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo up+replaying 2021-02-20T13:14:35.339 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep up+replaying 2021-02-20T13:14:35.341 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:up+replaying 2021-02-20T13:14:35.341 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:14:35.342 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:14:35.342 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap6 2021-02-20T13:14:35.342 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_snap cluster2 mirror test snap6 2021-02-20T13:14:35.342 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:14:35.342 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:14:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:14:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap6 2021-02-20T13:14:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap6 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:14:38.206 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5518796689532721) > weight (0.25) 2021-02-20T13:14:38.206 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7334915522506893) > weight (0.25) 2021-02-20T13:14:38.207 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3018273442894055) > weight (0.25) 2021-02-20T13:14:38.207 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6978509545193345) > weight (0.25) 2021-02-20T13:14:38.208 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T13:15:02.236 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T13:15:02.237 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:02.237 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:02.238 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:02.238 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T13:15:02.238 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:02.239 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:02.239 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:02.240 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state=started 2021-02-20T13:15:02.241 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:15:02.241 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:15:03.233 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T13:15:03.233 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:03.234 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:03.234 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:03.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local test_state=started 2021-02-20T13:15:03.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_result 2021-02-20T13:15:03.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_state=stopped 2021-02-20T13:15:03.236 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T13:15:03.236 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local+ cluster_instance=cluster1 2021-02-20T13:15:03.237 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:grep -i state 2021-02-20T13:15:03.237 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:15:03.237 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance=cluster1 2021-02-20T13:15:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loop_instance 2021-02-20T13:15:03.239 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0 2021-02-20T13:15:03.244 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T13:15:03.244 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 0 3 2021-02-20T13:15:03.244 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T13:15:03.245 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:0 2021-02-20T13:15:03.245 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:15:03.246 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:15:03.246 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T13:15:03.246 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:15:03.247 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:15:03.247 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:15:03.248 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:03.248 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:15:03.249 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:15:03.249 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:15:03.250 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:03.251 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:15:03.251 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:15:03.252 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T13:15:03.252 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:0 2021-02-20T13:15:03.252 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:03.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:15:03.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T13:15:03.254 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:15:03.255 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:15:03.255 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:15:03.255 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:03.256 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:15:03.256 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:15:03.256 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:15:03.257 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:15:03.257 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:15:03.257 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:15:03.258 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T13:15:03.259 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:15:03.260 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:15:03.260 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:15:03.262 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T13:15:03.371 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:15:03.372 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:15:03.372 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:15:03.372 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T13:15:03.372 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:1 2021-02-20T13:15:03.372 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:15:03.373 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:15:03.380 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T13:15:03.380 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:15:03.380 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:15:03.381 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:15:03.381 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:03.381 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:15:03.381 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:15:03.381 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:15:03.382 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:03.382 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:15:03.382 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:15:03.382 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T13:15:03.383 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:1 2021-02-20T13:15:03.383 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:03.383 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:15:03.383 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T13:15:03.383 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:15:03.384 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:15:03.384 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:15:03.384 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:03.384 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:15:03.384 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:15:03.385 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:15:03.385 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:15:03.385 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:15:03.385 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:15:03.385 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T13:15:03.391 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:15:03.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:15:03.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:15:03.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T13:15:03.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:15:03.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:15:03.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:15:03.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T13:15:03.502 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:2 2021-02-20T13:15:03.504 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:15:03.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:15:03.506 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T13:15:03.506 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:15:03.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:15:03.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:15:03.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:03.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:15:03.509 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:15:03.509 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:15:03.510 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:03.510 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:15:03.510 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:15:03.511 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T13:15:03.511 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:2 2021-02-20T13:15:03.512 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:03.512 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:15:03.512 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T13:15:03.513 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:15:03.513 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:15:03.513 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:15:03.514 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:03.515 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:15:03.516 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:15:03.516 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:15:03.516 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:15:03.517 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:15:03.517 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:15:03.517 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T13:15:03.526 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:15:03.527 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:15:03.527 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:15:03.527 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T13:15:03.613 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:15:03.613 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:15:03.613 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:15:03.615 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T13:15:03.615 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:3 2021-02-20T13:15:03.615 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:15:03.615 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:15:03.616 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T13:15:03.616 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:15:03.616 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:15:03.616 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:15:03.616 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:03.617 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:15:03.617 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:15:03.617 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:15:03.617 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:03.618 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:15:03.618 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:15:03.618 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T13:15:03.618 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:3 2021-02-20T13:15:03.618 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:03.619 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:15:03.619 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T13:15:03.619 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:15:03.619 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:15:03.619 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:15:03.620 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:03.620 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:15:03.620 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:15:03.620 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:15:03.621 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:15:03.621 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:15:03.621 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:15:03.621 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T13:15:03.637 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:15:03.637 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:15:03.638 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:15:03.638 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T13:15:03.755 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:15:03.755 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ status_result= "state": "Replaying" 2021-02-20T13:15:03.755 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo "state": "Replaying" 2021-02-20T13:15:03.756 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep -i Replaying 2021-02-20T13:15:03.758 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: "state": "Replaying" 2021-02-20T13:15:03.758 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_state=started 2021-02-20T13:15:03.758 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test started = started 2021-02-20T13:15:03.758 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:15:03.759 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T13:15:03.759 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:15:03.759 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:15:03.760 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:03.760 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:03.760 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = journal ] 2021-02-20T13:15:03.761 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = snapshot ] 2021-02-20T13:15:03.762 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T13:15:03.762 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:15:03.762 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:15:03.762 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:03.763 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:03.763 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T13:15:03.763 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster2-mirror-test.status 2021-02-20T13:15:03.763 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:15:03.764 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:15:03.764 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T13:15:03.765 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror-test.status 2021-02-20T13:15:03.765 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:15:03.765 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:15:03.765 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T13:15:03.766 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:15:03.766 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:03.766 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:03.766 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T13:15:04.257 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Snapshot ID: 26 2021-02-20T13:15:04.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:15:04.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:15:04.269 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:04.269 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:04.269 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:15:04.269 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T13:15:04.270 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:15:04.330 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T13:15:04.335 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snapshot_id=26 2021-02-20T13:15:04.336 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:15:04.336 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.2 2021-02-20T13:15:04.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:15:04.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:04.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:04.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:04.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:15:04.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:15:04.539 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:15:04.611 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:15:04.615 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id=22 2021-02-20T13:15:04.615 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 26 = 22 2021-02-20T13:15:04.616 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.4 2021-02-20T13:15:05.017 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:15:05.018 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:05.019 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:05.019 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:05.019 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:15:05.020 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:15:05.020 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:15:05.087 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:15:05.087 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:15:05.092 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:15:05.092 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:15:05.092 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:15:05.093 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:15:05.094 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 26 = 2021-02-20T13:15:05.094 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.8 2021-02-20T13:15:05.895 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:15:05.895 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:05.895 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:05.896 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:05.896 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:15:05.896 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:15:05.897 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:15:05.966 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:15:05.967 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:15:05.971 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:15:05.972 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:15:05.972 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:15:05.973 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:15:05.973 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 26 = 2021-02-20T13:15:05.973 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1.6 2021-02-20T13:15:07.575 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:15:07.575 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:07.576 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:07.576 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:07.577 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:15:07.577 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:15:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:15:07.641 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:15:07.642 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:15:07.643 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:15:07.644 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:15:07.644 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:15:07.644 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:15:07.644 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 26 = 2021-02-20T13:15:07.645 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 2 2021-02-20T13:15:09.645 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:15:09.646 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:09.646 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:09.646 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:09.646 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:15:09.647 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:15:09.647 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:15:09.844 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:15:09.845 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:15:09.847 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:15:09.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:15:09.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:15:09.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:15:09.849 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 26 = 2021-02-20T13:15:09.849 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 2 2021-02-20T13:15:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:15:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:11.851 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:15:11.851 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:15:11.851 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:15:11.922 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:15:11.928 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id=26 2021-02-20T13:15:11.928 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 26 = 26 2021-02-20T13:15:11.928 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:15:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snap_present cluster1 mirror test snap6 2021-02-20T13:15:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:11.930 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap6 2021-02-20T13:15:11.930 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:15:11.930 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:15:12.929 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 info mirror/test@snap6 2021-02-20T13:15:13.001 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:rbd image 'test': 2021-02-20T13:15:13.001 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: size 512 MiB in 128 objects 2021-02-20T13:15:13.001 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: order 22 (4 MiB objects) 2021-02-20T13:15:13.002 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: snapshot_count: 12 2021-02-20T13:15:13.002 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: id: 109d36d23f3 2021-02-20T13:15:13.002 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: block_name_prefix: rbd_data.109d36d23f3 2021-02-20T13:15:13.003 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: format: 2 2021-02-20T13:15:13.003 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: features: layering, exclusive-lock, non-primary 2021-02-20T13:15:13.003 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: op_features: 2021-02-20T13:15:13.003 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: flags: 2021-02-20T13:15:13.003 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: create_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:15:13.004 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: access_timestamp: Sat Feb 20 13:13:26 2021 2021-02-20T13:15:13.004 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: modify_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:15:13.004 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: protected: False 2021-02-20T13:15:13.005 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring state: enabled 2021-02-20T13:15:13.005 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring mode: snapshot 2021-02-20T13:15:13.005 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring global id: 49a0b5eb-7d69-4824-806b-1ece12354eb9 2021-02-20T13:15:13.005 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring primary: false 2021-02-20T13:15:13.007 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:15:13.007 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snap5-clean ] 2021-02-20T13:15:13.007 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap5-clean 2021-02-20T13:15:13.007 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:13.008 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:13.008 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap5-clean 2021-02-20T13:15:13.009 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:15:13.009 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:15:13.009 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:15:13.009 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:15:13.010 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5-clean /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:15:13.843 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:15:13.847 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5-clean /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:15:14.876 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:15:14.880 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:15:15.157 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:15:15.289 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:15:15.289 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap6 2021-02-20T13:15:15.289 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:15.290 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:15.290 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap6 2021-02-20T13:15:15.291 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:15:15.291 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:15:15.291 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:15:15.292 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:15:15.292 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6 /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:15:16.104 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:15:16.108 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6 /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:15:16.658 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T13:15:16.648+0000 7fb8139c0700 -1 bdev(0x557840622000 /var/lib/ceph/osd/cluster1-2/block) aio_submit retries 1 2021-02-20T13:15:16.658 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T13:15:16.648+0000 7fb8159c4700 -1 bdev(0x557840622000 /var/lib/ceph/osd/cluster1-2/block) aio_submit retries 1 2021-02-20T13:15:17.078 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:15:17.082 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:15:17.363 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:15:17.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:15:17.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ clean_snap_name=snap6-clean 2021-02-20T13:15:17.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_snap cluster2 mirror test snap6-clean 2021-02-20T13:15:17.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:15:17.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:17.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:17.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap6-clean 2021-02-20T13:15:17.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap6-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T13:15:18.236 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T13:15:18.236 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:15:18.237 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:18.237 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:18.237 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:15:18.238 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=7 2021-02-20T13:15:18.238 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:15:18.238 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T13:15:19.829 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 89185280 bytes 2021-02-20T13:15:20.238 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 96916480 bytes 2021-02-20T13:15:20.852 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 163244032 bytes 2021-02-20T13:15:21.169 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 200415232 bytes 2021-02-20T13:15:21.724 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 268790784 bytes 2021-02-20T13:15:21.957 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 308685824 bytes 2021-02-20T13:15:22.477 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 346401792 bytes 2021-02-20T13:15:22.920 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 405568512 bytes 2021-02-20T13:15:23.041 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 407176192 bytes 2021-02-20T13:15:23.234 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 419395584 bytes 2021-02-20T13:15:23.569 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 464036864 bytes 2021-02-20T13:15:24.070 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 534434816 bytes 2021-02-20T13:15:24.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 563396608 bytes 2021-02-20T13:15:24.710 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 600092672 bytes 2021-02-20T13:15:24.911 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 609817600 bytes 2021-02-20T13:15:25.030 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 610897920 bytes 2021-02-20T13:15:25.246 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:15:25.247 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:15:25.248 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:15:25.249 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:15:25.250 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T13:15:25.250 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:25.251 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:25.251 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:25.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:15:25.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local description_pattern= 2021-02-20T13:15:25.254 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local service_pattern= 2021-02-20T13:15:25.254 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:15:26.245 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T13:15:26.246 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:26.246 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:26.247 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:26.247 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:15:26.247 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local description_pattern= 2021-02-20T13:15:26.248 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local service_pattern= 2021-02-20T13:15:26.248 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T13:15:26.248 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T13:15:26.249 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:15:26.251 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:15:26.251 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T13:15:26.252 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ tee /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:15:26.322 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:test: 2021-02-20T13:15:26.322 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: global_id: 49a0b5eb-7d69-4824-806b-1ece12354eb9 2021-02-20T13:15:26.323 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state: up+replaying 2021-02-20T13:15:26.323 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: description: replaying, {"bytes_per_second":15563946.67,"bytes_per_snapshot":422123008.0,"local_snapshot_timestamp":1613826904,"remote_snapshot_timestamp":1613826904,"replay_state":"idle"} 2021-02-20T13:15:26.324 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: service: mirror.3 on smithi183 2021-02-20T13:15:26.324 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: last_update: 2021-02-20 13:15:25 2021-02-20T13:15:26.325 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: peer_sites: 2021-02-20T13:15:26.325 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: name: cluster2 2021-02-20T13:15:26.326 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state: up+stopped 2021-02-20T13:15:26.326 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: description: local image is primary 2021-02-20T13:15:26.326 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: last_update: 2021-02-20 13:15:25 2021-02-20T13:15:26.332 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:15:26.334 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: state: up+replaying 2021-02-20T13:15:26.334 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^ description: .* /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:15:26.345 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: description: replaying, {"bytes_per_second":15563946.67,"bytes_per_snapshot":422123008.0,"local_snapshot_timestamp":1613826904,"remote_snapshot_timestamp":1613826904,"replay_state":"idle"} 2021-02-20T13:15:26.345 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:up+replaying 2021-02-20T13:15:26.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: service: mirror.3 on smithi183 2021-02-20T13:15:26.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:15:26.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo up+replaying 2021-02-20T13:15:26.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^up+ 2021-02-20T13:15:26.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep service: *mirror..* on /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:15:26.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:15:26.353 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local last_update=2021-02-20 13:15:25 2021-02-20T13:15:26.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 13:15:25 2021-02-20T13:15:26.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:26.355 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:26.355 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:26.356 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:15:26.357 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local prev_last_update=2021-02-20 13:15:25 2021-02-20T13:15:26.358 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T13:15:26.358 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T13:15:26.359 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:15:26.360 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror.mirror_status 2021-02-20T13:15:26.360 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T13:15:26.389 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T13:15:26.473 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local last_update state 2021-02-20T13:15:26.473 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T13:15:26.479 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ last_update=2021-02-20 13:15:25 2021-02-20T13:15:26.479 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T13:15:26.484 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ state=up+replaying 2021-02-20T13:15:26.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo up+replaying 2021-02-20T13:15:26.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep up+replaying 2021-02-20T13:15:26.486 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:up+replaying 2021-02-20T13:15:26.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:15:26.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:15:26.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap7 2021-02-20T13:15:26.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_snap cluster2 mirror test snap7 2021-02-20T13:15:26.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:15:26.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:26.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:26.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap7 2021-02-20T13:15:26.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap7 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:15:56.269 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T13:15:56.270 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:56.270 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:56.271 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:56.271 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T13:15:56.272 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:56.273 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:56.274 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:56.274 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state=started 2021-02-20T13:15:56.275 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:15:56.275 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:15:57.274 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T13:15:57.274 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:57.275 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:57.275 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:57.275 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local test_state=started 2021-02-20T13:15:57.276 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_result 2021-02-20T13:15:57.276 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_state=stopped 2021-02-20T13:15:57.276 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T13:15:57.276 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_instance=cluster1 2021-02-20T13:15:57.277 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ + shift 2021-02-20T13:15:57.277 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ localgrep cluster=cluster1 -i 2021-02-20T13:15:57.277 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state 2021-02-20T13:15:57.277 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance=cluster1 2021-02-20T13:15:57.278 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loop_instance 2021-02-20T13:15:57.278 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0 2021-02-20T13:15:57.280 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T13:15:57.280 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 0 3 2021-02-20T13:15:57.280 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T13:15:57.281 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:0 2021-02-20T13:15:57.281 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:15:57.281 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:15:57.281 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T13:15:57.282 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:15:57.282 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:15:57.282 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:15:57.282 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:57.283 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:15:57.283 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:15:57.283 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:15:57.283 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:57.283 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:15:57.284 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:15:57.284 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T13:15:57.284 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:0 2021-02-20T13:15:57.284 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:57.284 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:15:57.285 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T13:15:57.285 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:15:57.285 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:15:57.286 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:15:57.286 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:57.287 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:15:57.287 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:15:57.287 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:15:57.287 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:15:57.288 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:15:57.288 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:15:57.288 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T13:15:57.299 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:15:57.300 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:15:57.300 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:15:57.300 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T13:15:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:15:57.405 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:15:57.405 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:15:57.406 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T13:15:57.407 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:1 2021-02-20T13:15:57.407 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:15:57.407 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:15:57.407 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T13:15:57.408 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:15:57.408 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:15:57.408 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:15:57.408 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:57.409 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:15:57.409 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:15:57.409 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:15:57.409 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:57.409 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:15:57.410 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:15:57.410 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T13:15:57.410 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:1 2021-02-20T13:15:57.410 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:57.411 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:15:57.411 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T13:15:57.411 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:15:57.411 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:15:57.412 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:15:57.412 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:57.412 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:15:57.412 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:15:57.413 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:15:57.413 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:15:57.413 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:15:57.414 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:15:57.414 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T13:15:57.427 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:15:57.427 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:15:57.427 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:15:57.427 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T13:15:57.531 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:15:57.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:15:57.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:15:57.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T13:15:57.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:2 2021-02-20T13:15:57.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:15:57.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:15:57.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T13:15:57.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:15:57.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:15:57.534 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:15:57.534 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:57.534 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:15:57.534 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:15:57.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:15:57.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:57.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:15:57.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:15:57.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T13:15:57.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:2 2021-02-20T13:15:57.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:57.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:15:57.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T13:15:57.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:15:57.539 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:15:57.539 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:15:57.539 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:57.540 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:15:57.540 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:15:57.540 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:15:57.540 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:15:57.541 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:15:57.541 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:15:57.541 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T13:15:57.553 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:15:57.553 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:15:57.554 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:15:57.554 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T13:15:57.651 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:15:57.651 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:15:57.652 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:15:57.655 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T13:15:57.655 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:3 2021-02-20T13:15:57.655 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:15:57.656 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:15:57.656 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T13:15:57.656 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:15:57.656 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:15:57.656 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:15:57.657 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:57.657 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:15:57.657 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:15:57.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:15:57.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:57.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:15:57.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:15:57.659 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T13:15:57.659 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:3 2021-02-20T13:15:57.659 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:57.660 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:15:57.660 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T13:15:57.660 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:15:57.661 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:15:57.661 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:15:57.661 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:15:57.662 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:15:57.662 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:15:57.662 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:15:57.663 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:15:57.663 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:15:57.664 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:15:57.664 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T13:15:57.674 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:15:57.674 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:15:57.676 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:15:57.677 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T13:15:57.782 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:15:57.782 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ status_result= "state": "Replaying" 2021-02-20T13:15:57.783 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo "state": "Replaying" 2021-02-20T13:15:57.783 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep -i Replaying 2021-02-20T13:15:57.785 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: "state": "Replaying" 2021-02-20T13:15:57.785 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_state=started 2021-02-20T13:15:57.785 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test started = started 2021-02-20T13:15:57.786 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:15:57.786 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T13:15:57.786 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:15:57.787 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:15:57.787 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:57.787 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:57.787 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = journal ] 2021-02-20T13:15:57.788 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = snapshot ] 2021-02-20T13:15:57.788 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T13:15:57.788 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:15:57.788 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:15:57.788 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:57.789 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:57.789 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T13:15:57.789 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster2-mirror-test.status 2021-02-20T13:15:57.789 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:15:57.791 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:15:57.792 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T13:15:57.792 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror-test.status 2021-02-20T13:15:57.792 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:15:57.792 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:15:57.793 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T13:15:57.793 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:15:57.793 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:57.793 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:57.793 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T13:15:58.137 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:15:58.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:15:58.138 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T13:15:58.138 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:15:58.138 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:15:58.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T13:15:58.134+0000 7fb149975700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 23551) UID: 1000 2021-02-20T13:15:58.297 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Snapshot ID: 30 2021-02-20T13:15:58.314 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:15:58.314 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:15:58.315 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:58.315 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:58.315 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:15:58.315 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T13:15:58.315 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:15:58.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:15:58.367 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T13:15:58.370 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snapshot_id=30 2021-02-20T13:15:58.371 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:15:58.371 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.2 2021-02-20T13:15:58.572 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:15:58.573 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:58.573 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:58.573 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:58.573 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:15:58.574 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:15:58.574 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:15:58.650 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:15:58.650 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:15:58.654 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:15:58.654 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:15:58.655 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:15:58.655 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:15:58.656 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 30 = 2021-02-20T13:15:58.656 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.4 2021-02-20T13:15:59.056 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:15:59.057 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:59.057 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:59.057 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:59.057 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:15:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:15:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:15:59.131 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:15:59.132 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:15:59.135 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:15:59.135 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:15:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:15:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:15:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 30 = 2021-02-20T13:15:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.8 2021-02-20T13:15:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:15:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:15:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:15:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:15:59.939 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:15:59.939 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:15:59.939 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:16:00.014 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:16:00.014 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:16:00.018 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:16:00.018 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:16:00.019 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:16:00.019 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:16:00.020 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 30 = 2021-02-20T13:16:00.020 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1.6 2021-02-20T13:16:01.623 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:16:01.623 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:16:01.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:01.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:01.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:16:01.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:16:01.625 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:16:02.150 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:16:02.150 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:16:02.155 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:16:02.156 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:16:02.156 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:16:02.157 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:16:02.157 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 30 = 2021-02-20T13:16:02.158 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 2 2021-02-20T13:16:04.157 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:16:04.158 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:16:04.158 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:04.158 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:04.159 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:16:04.159 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:16:04.159 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:16:04.229 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:16:04.234 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id=30 2021-02-20T13:16:04.234 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 30 = 30 2021-02-20T13:16:04.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:16:04.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snap_present cluster1 mirror test snap7 2021-02-20T13:16:04.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:16:04.236 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:04.236 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:04.237 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap7 2021-02-20T13:16:04.237 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:16:04.238 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:16:04.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:16:04.242 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:16:04.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:16:04.243 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:16:04.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:16:04.249 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T13:16:05.210 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8800189791691819) > weight (0.25) 2021-02-20T13:16:05.211 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9733481812479453) > weight (0.25) 2021-02-20T13:16:05.211 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7154847076477464) > weight (0.25) 2021-02-20T13:16:05.211 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T13:16:05.211 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T13:16:05.212 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T13:16:05.219 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T13:16:05.209+0000 7f884ec78700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 19237) UID: 1000 2021-02-20T13:16:05.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 info mirror/test@snap7 2021-02-20T13:16:05.296 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:rbd image 'test': 2021-02-20T13:16:05.297 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: size 512 MiB in 128 objects 2021-02-20T13:16:05.297 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: order 22 (4 MiB objects) 2021-02-20T13:16:05.298 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: snapshot_count: 14 2021-02-20T13:16:05.298 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: id: 109d36d23f3 2021-02-20T13:16:05.298 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: block_name_prefix: rbd_data.109d36d23f3 2021-02-20T13:16:05.299 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: format: 2 2021-02-20T13:16:05.299 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: features: layering, exclusive-lock, non-primary 2021-02-20T13:16:05.300 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: op_features: 2021-02-20T13:16:05.300 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: flags: 2021-02-20T13:16:05.300 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: create_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:16:05.301 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: access_timestamp: Sat Feb 20 13:15:13 2021 2021-02-20T13:16:05.301 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: modify_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:16:05.302 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: protected: False 2021-02-20T13:16:05.302 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring state: enabled 2021-02-20T13:16:05.302 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring mode: snapshot 2021-02-20T13:16:05.303 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring global id: 49a0b5eb-7d69-4824-806b-1ece12354eb9 2021-02-20T13:16:05.305 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring primary: false 2021-02-20T13:16:05.307 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:16:05.308 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snap6-clean ] 2021-02-20T13:16:05.308 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap6-clean 2021-02-20T13:16:05.308 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:05.309 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:05.309 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap6-clean 2021-02-20T13:16:05.310 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:16:05.310 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:16:05.311 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:16:05.311 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:16:05.312 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6-clean /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:16:05.415 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:16:06.126 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:16:06.129 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6-clean /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:16:07.111 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:16:07.115 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:16:07.391 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:16:07.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:16:07.519 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap7 2021-02-20T13:16:07.519 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:07.520 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:07.520 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap7 2021-02-20T13:16:07.521 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:16:07.521 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:16:07.522 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:16:07.522 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:16:07.523 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7 /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:16:08.345 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:16:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7 /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:16:09.378 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:16:09.382 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:16:09.659 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:16:09.787 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:16:09.787 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ clean_snap_name=snap7-clean 2021-02-20T13:16:09.787 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_snap cluster2 mirror test snap7-clean 2021-02-20T13:16:09.788 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:16:09.788 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:09.788 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:09.788 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap7-clean 2021-02-20T13:16:09.789 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap7-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T13:16:10.173 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T13:16:10.174 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:16:10.174 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:10.174 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:10.175 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:16:10.176 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=10 2021-02-20T13:16:10.176 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:16:10.176 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T13:16:11.128 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 17548288 bytes 2021-02-20T13:16:11.635 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 35569664 bytes 2021-02-20T13:16:12.547 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 139090944 bytes 2021-02-20T13:16:12.959 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 174046208 bytes 2021-02-20T13:16:13.213 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T13:16:13.214 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:16:13.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:16:13.215 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T13:16:13.215 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T13:16:13.215 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T13:16:13.217 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T13:16:13.218 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T13:16:13.345 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 211476480 bytes 2021-02-20T13:16:13.597 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 231360512 bytes 2021-02-20T13:16:14.280 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 317572096 bytes 2021-02-20T13:16:14.574 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 336683008 bytes 2021-02-20T13:16:14.787 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 359160832 bytes 2021-02-20T13:16:15.001 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 367382528 bytes 2021-02-20T13:16:15.228 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 393297920 bytes 2021-02-20T13:16:15.691 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 440910848 bytes 2021-02-20T13:16:15.950 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 460115968 bytes 2021-02-20T13:16:16.411 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 539242496 bytes 2021-02-20T13:16:16.862 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 569211904 bytes 2021-02-20T13:16:17.198 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 601123840 bytes 2021-02-20T13:16:17.460 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 619859968 bytes 2021-02-20T13:16:17.809 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 670842880 bytes 2021-02-20T13:16:18.237 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 707085312 bytes 2021-02-20T13:16:18.740 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 749803520 bytes 2021-02-20T13:16:19.257 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 825188352 bytes 2021-02-20T13:16:19.756 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 891897856 bytes 2021-02-20T13:16:20.181 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:16:20.181 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:16:20.182 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:16:20.182 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:16:20.182 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T13:16:20.182 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:16:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:16:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local description_pattern= 2021-02-20T13:16:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local service_pattern= 2021-02-20T13:16:20.184 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:16:21.186 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T13:16:21.186 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:16:21.187 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:21.187 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:21.187 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:16:21.188 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local description_pattern= 2021-02-20T13:16:21.188 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local service_pattern= 2021-02-20T13:16:21.189 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T13:16:21.189 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T13:16:21.190 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:16:21.190 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:16:21.191 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T13:16:21.191 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ tee /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:16:21.267 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:test: 2021-02-20T13:16:21.267 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: global_id: 49a0b5eb-7d69-4824-806b-1ece12354eb9 2021-02-20T13:16:21.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state: up+replaying 2021-02-20T13:16:21.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":422123008.0,"local_snapshot_timestamp":1613826904,"remote_snapshot_timestamp":1613826904,"replay_state":"idle"} 2021-02-20T13:16:21.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: service: mirror.3 on smithi183 2021-02-20T13:16:21.269 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: last_update: 2021-02-20 13:15:55 2021-02-20T13:16:21.269 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: peer_sites: 2021-02-20T13:16:21.270 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: name: cluster2 2021-02-20T13:16:21.270 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state: up+stopped 2021-02-20T13:16:21.271 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: description: local image is primary 2021-02-20T13:16:21.271 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: last_update: 2021-02-20 13:15:55 2021-02-20T13:16:21.277 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:16:21.278 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: state: up+replaying 2021-02-20T13:16:21.279 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^ description: .* /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:16:21.298 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":422123008.0,"local_snapshot_timestamp":1613826904,"remote_snapshot_timestamp":1613826904,"replay_state":"idle"} 2021-02-20T13:16:21.299 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:up+replaying 2021-02-20T13:16:21.299 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: service: mirror.3 on smithi183 2021-02-20T13:16:21.300 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:16:21.300 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo up+replaying 2021-02-20T13:16:21.300 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^up+ 2021-02-20T13:16:21.301 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep service: *mirror..* on /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:16:21.301 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:16:21.302 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local last_update=2021-02-20 13:15:55 2021-02-20T13:16:21.302 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 13:15:55 2021-02-20T13:16:21.302 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:16:21.303 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:21.303 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:21.303 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:16:21.304 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local prev_last_update=2021-02-20 13:15:55 2021-02-20T13:16:21.304 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T13:16:21.305 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T13:16:21.305 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:16:21.305 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror.mirror_status 2021-02-20T13:16:21.306 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T13:16:21.336 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T13:16:21.449 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local last_update state 2021-02-20T13:16:21.449 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T13:16:21.454 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ last_update=2021-02-20 13:15:55 2021-02-20T13:16:21.455 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T13:16:21.459 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ state=up+replaying 2021-02-20T13:16:21.460 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo up+replaying 2021-02-20T13:16:21.460 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep up+replaying 2021-02-20T13:16:21.461 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:up+replaying 2021-02-20T13:16:21.462 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:16:21.462 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:16:21.462 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap8 2021-02-20T13:16:21.463 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_snap cluster2 mirror test snap8 2021-02-20T13:16:21.463 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:16:21.463 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:21.463 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:21.463 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap8 2021-02-20T13:16:21.464 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap8 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:16:51.302 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T13:16:51.302 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:16:51.303 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:51.303 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:51.303 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T13:16:51.303 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:16:51.303 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:51.304 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:51.304 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state=started 2021-02-20T13:16:51.304 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:16:51.304 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:16:52.303 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T13:16:52.303 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:16:52.303 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:52.304 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:52.304 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local test_state=started 2021-02-20T13:16:52.304 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_result 2021-02-20T13:16:52.304 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_state=stopped 2021-02-20T13:16:52.305 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T13:16:52.305 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_instance=cluster1 2021-02-20T13:16:52.306 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:16:52.306 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ + local cluster=cluster1 2021-02-20T13:16:52.306 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ localgrep instance=cluster1 -i 2021-02-20T13:16:52.306 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state+ 2021-02-20T13:16:52.307 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:local loop_instance 2021-02-20T13:16:52.307 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0 2021-02-20T13:16:52.308 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T13:16:52.308 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 0 3 2021-02-20T13:16:52.310 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T13:16:52.311 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:0 2021-02-20T13:16:52.311 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:16:52.311 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:16:52.311 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T13:16:52.312 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:16:52.312 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:16:52.313 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:16:52.314 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:16:52.314 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:16:52.314 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:16:52.314 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:16:52.315 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:16:52.315 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:16:52.315 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:16:52.315 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T13:16:52.315 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:0 2021-02-20T13:16:52.316 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:16:52.316 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:16:52.316 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T13:16:52.316 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:16:52.316 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:16:52.317 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:16:52.317 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:16:52.317 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:16:52.317 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:16:52.318 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:16:52.318 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:16:52.318 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:16:52.318 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:16:52.318 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T13:16:52.329 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:16:52.329 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:16:52.329 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:16:52.330 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T13:16:52.436 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:16:52.437 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:16:52.437 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:16:52.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T13:16:52.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:1 2021-02-20T13:16:52.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:16:52.440 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:16:52.440 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T13:16:52.440 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:16:52.440 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:16:52.440 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:16:52.441 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:16:52.441 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:16:52.441 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:16:52.441 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:16:52.441 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:16:52.442 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:16:52.442 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:16:52.443 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T13:16:52.443 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:1 2021-02-20T13:16:52.443 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:16:52.443 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:16:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T13:16:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:16:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:16:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:16:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:16:52.445 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:16:52.445 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:16:52.445 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:16:52.445 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:16:52.445 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:16:52.446 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:16:52.446 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T13:16:52.460 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:16:52.460 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:16:52.461 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:16:52.461 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T13:16:52.558 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:16:52.558 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:16:52.559 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:16:52.560 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T13:16:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:2 2021-02-20T13:16:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:16:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:16:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T13:16:52.562 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:16:52.564 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:16:52.564 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:16:52.564 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:16:52.565 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:16:52.565 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:16:52.565 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:16:52.565 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:16:52.566 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:16:52.566 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:16:52.566 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T13:16:52.566 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:2 2021-02-20T13:16:52.567 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:16:52.567 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:16:52.567 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T13:16:52.567 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:16:52.568 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:16:52.568 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:16:52.568 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:16:52.568 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:16:52.568 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:16:52.569 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:16:52.569 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:16:52.569 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:16:52.569 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:16:52.569 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T13:16:52.583 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:16:52.583 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:16:52.583 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:16:52.583 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T13:16:52.675 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:16:52.675 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:16:52.675 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:16:52.677 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T13:16:52.677 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:3 2021-02-20T13:16:52.677 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:16:52.677 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:16:52.678 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T13:16:52.678 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:16:52.678 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:16:52.678 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:16:52.679 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:16:52.679 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:16:52.679 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:16:52.679 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:16:52.680 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:16:52.680 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:16:52.681 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:16:52.682 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T13:16:52.682 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:3 2021-02-20T13:16:52.684 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:16:52.684 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:16:52.684 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T13:16:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:16:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:16:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:16:52.686 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:16:52.686 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:16:52.686 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:16:52.686 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:16:52.687 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:16:52.687 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:16:52.687 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:16:52.687 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T13:16:52.699 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:16:52.699 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:16:52.699 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:16:52.700 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T13:16:52.811 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:16:52.814 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ status_result= "state": "Replaying" 2021-02-20T13:16:52.814 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ + echo "state": "Replaying" 2021-02-20T13:16:52.815 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:grep -i Replaying 2021-02-20T13:16:52.815 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_state=started 2021-02-20T13:16:52.815 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test started = started 2021-02-20T13:16:52.816 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:16:52.816 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T13:16:52.817 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:16:52.817 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:16:52.817 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:52.817 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:52.818 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = journal ] 2021-02-20T13:16:52.818 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = snapshot ] 2021-02-20T13:16:52.818 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T13:16:52.819 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:16:52.819 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:16:52.819 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:52.820 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:52.820 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T13:16:52.820 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster2-mirror-test.status 2021-02-20T13:16:52.821 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:16:52.821 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: "state": "Replaying" 2021-02-20T13:16:52.823 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:16:52.823 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T13:16:52.823 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror-test.status 2021-02-20T13:16:52.826 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:16:52.827 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:16:52.827 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T13:16:52.827 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:16:52.828 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:52.828 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:52.830 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T13:16:53.331 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Snapshot ID: 34 2021-02-20T13:16:53.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:16:53.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:16:53.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:53.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:53.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:16:53.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T13:16:53.350 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:16:53.415 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T13:16:53.417 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snapshot_id=34 2021-02-20T13:16:53.418 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:16:53.418 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.2 2021-02-20T13:16:53.619 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:16:53.620 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:16:53.620 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:53.621 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:53.621 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:16:53.622 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:16:53.622 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:16:53.696 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:16:53.706 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id=30 2021-02-20T13:16:53.707 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 34 = 30 2021-02-20T13:16:53.707 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.4 2021-02-20T13:16:54.104 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:16:54.104 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:16:54.105 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:54.105 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:54.105 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:16:54.105 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:16:54.105 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:16:54.180 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:16:54.180 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:16:54.185 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:16:54.185 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:16:54.185 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:16:54.186 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:16:54.186 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 34 = 2021-02-20T13:16:54.186 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.8 2021-02-20T13:16:54.987 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:16:54.988 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:16:54.988 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:54.988 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:54.988 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:16:54.989 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:16:54.989 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:16:55.062 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:16:55.062 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:16:55.066 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:16:55.066 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:16:55.066 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:16:55.067 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:16:55.067 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 34 = 2021-02-20T13:16:55.067 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1.6 2021-02-20T13:16:56.668 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:16:56.668 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:16:56.669 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:56.669 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:56.669 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:16:56.669 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:16:56.669 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:16:56.753 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:16:56.754 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:16:56.758 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:16:56.758 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:16:56.758 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:16:56.759 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:16:56.759 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 34 = 2021-02-20T13:16:56.759 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 2 2021-02-20T13:16:58.760 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:16:58.760 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:16:58.760 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:16:58.761 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:16:58.761 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:16:58.761 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:16:58.761 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:16:58.930 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:16:58.931 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:16:58.935 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:16:58.935 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:16:58.935 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:16:58.936 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:16:58.936 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 34 = 2021-02-20T13:16:58.936 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 2 2021-02-20T13:17:00.938 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:17:00.939 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:17:00.939 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:00.940 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:00.940 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:17:00.941 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:17:00.941 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:17:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:17:01.020 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id=34 2021-02-20T13:17:01.021 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 34 = 34 2021-02-20T13:17:01.021 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:17:01.022 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snap_present cluster1 mirror test snap8 2021-02-20T13:17:01.022 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:17:01.023 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:01.023 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:01.023 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap8 2021-02-20T13:17:01.024 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:17:01.024 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:17:02.021 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 info mirror/test@snap8 2021-02-20T13:17:02.087 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:rbd image 'test': 2021-02-20T13:17:02.087 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: size 512 MiB in 128 objects 2021-02-20T13:17:02.088 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: order 22 (4 MiB objects) 2021-02-20T13:17:02.088 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: snapshot_count: 16 2021-02-20T13:17:02.088 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: id: 109d36d23f3 2021-02-20T13:17:02.088 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: block_name_prefix: rbd_data.109d36d23f3 2021-02-20T13:17:02.089 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: format: 2 2021-02-20T13:17:02.089 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: features: layering, exclusive-lock, non-primary 2021-02-20T13:17:02.089 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: op_features: 2021-02-20T13:17:02.089 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: flags: 2021-02-20T13:17:02.090 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: create_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:17:02.090 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: access_timestamp: Sat Feb 20 13:15:13 2021 2021-02-20T13:17:02.090 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: modify_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:17:02.090 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: protected: False 2021-02-20T13:17:02.090 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring state: enabled 2021-02-20T13:17:02.091 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring mode: snapshot 2021-02-20T13:17:02.091 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring global id: 49a0b5eb-7d69-4824-806b-1ece12354eb9 2021-02-20T13:17:02.091 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring primary: false 2021-02-20T13:17:02.092 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:17:02.093 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snap7-clean ] 2021-02-20T13:17:02.093 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap7-clean 2021-02-20T13:17:02.093 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:02.093 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:02.094 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap7-clean 2021-02-20T13:17:02.094 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:17:02.094 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:17:02.094 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:17:02.095 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:17:02.095 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7-clean /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:17:02.935 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:17:02.938 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7-clean /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:17:03.869 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:17:03.873 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:17:04.153 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:17:04.281 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:17:04.281 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap8 2021-02-20T13:17:04.281 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:04.281 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:04.282 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap8 2021-02-20T13:17:04.282 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:17:04.282 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:17:04.282 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:17:04.282 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:17:04.283 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8 /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:17:05.088 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:17:05.091 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8 /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:17:06.125 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:17:06.129 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:17:06.410 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:17:06.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:17:06.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ clean_snap_name=snap8-clean 2021-02-20T13:17:06.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_snap cluster2 mirror test snap8-clean 2021-02-20T13:17:06.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:17:06.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:06.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:06.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap8-clean 2021-02-20T13:17:06.539 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap8-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T13:17:07.299 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T13:17:07.300 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:17:07.300 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:07.300 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:07.300 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:17:07.301 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=9 2021-02-20T13:17:07.302 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:17:07.302 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T13:17:08.372 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 45146112 bytes 2021-02-20T13:17:09.525 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 157565952 bytes 2021-02-20T13:17:09.938 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 180435968 bytes 2021-02-20T13:17:10.511 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 246531072 bytes 2021-02-20T13:17:10.879 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 278566912 bytes 2021-02-20T13:17:11.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 331815936 bytes 2021-02-20T13:17:11.605 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 350380032 bytes 2021-02-20T13:17:12.165 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 425295872 bytes 2021-02-20T13:17:12.445 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 440695808 bytes 2021-02-20T13:17:12.930 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 511703040 bytes 2021-02-20T13:17:13.502 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 565688320 bytes 2021-02-20T13:17:13.965 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 616826880 bytes 2021-02-20T13:17:14.511 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 673672192 bytes 2021-02-20T13:17:14.887 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 715958272 bytes 2021-02-20T13:17:15.104 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 755986432 bytes 2021-02-20T13:17:15.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 775956480 bytes 2021-02-20T13:17:15.837 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 807725056 bytes 2021-02-20T13:17:16.243 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 851164160 bytes 2021-02-20T13:17:16.310 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:17:16.311 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:17:16.311 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:17:16.311 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:17:16.312 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T13:17:16.312 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:17:16.312 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:16.312 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:16.312 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:17:16.313 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local description_pattern= 2021-02-20T13:17:16.313 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local service_pattern= 2021-02-20T13:17:16.313 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:17:17.319 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T13:17:17.320 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:17:17.320 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:17.320 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:17.321 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:17:17.321 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local description_pattern= 2021-02-20T13:17:17.321 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local service_pattern= 2021-02-20T13:17:17.321 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T13:17:17.321 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T13:17:17.322 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:17:17.323 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:17:17.323 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T13:17:17.323 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ tee /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:17:17.405 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:test: 2021-02-20T13:17:17.406 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: global_id: 49a0b5eb-7d69-4824-806b-1ece12354eb9 2021-02-20T13:17:17.406 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state: up+replaying 2021-02-20T13:17:17.406 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":419834880.0,"local_snapshot_timestamp":1613826958,"remote_snapshot_timestamp":1613826958,"replay_state":"idle"} 2021-02-20T13:17:17.406 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: service: mirror.3 on smithi183 2021-02-20T13:17:17.407 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: last_update: 2021-02-20 13:16:55 2021-02-20T13:17:17.407 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: peer_sites: 2021-02-20T13:17:17.407 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: name: cluster2 2021-02-20T13:17:17.407 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state: up+stopped 2021-02-20T13:17:17.408 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: description: local image is primary 2021-02-20T13:17:17.408 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: last_update: 2021-02-20 13:16:55 2021-02-20T13:17:17.415 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:17:17.417 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: state: up+replaying 2021-02-20T13:17:17.417 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^ description: .* /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:17:17.419 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":419834880.0,"local_snapshot_timestamp":1613826958,"remote_snapshot_timestamp":1613826958,"replay_state":"idle"} 2021-02-20T13:17:17.419 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:17:17.420 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo up+replaying 2021-02-20T13:17:17.420 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^up+ 2021-02-20T13:17:17.421 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:up+replaying 2021-02-20T13:17:17.421 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep service: *mirror..* on /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:17:17.422 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: service: mirror.3 on smithi183 2021-02-20T13:17:17.423 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:17:17.427 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local last_update=2021-02-20 13:16:55 2021-02-20T13:17:17.428 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 13:16:55 2021-02-20T13:17:17.428 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:17:17.428 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:17.428 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:17.428 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:17:17.429 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local prev_last_update=2021-02-20 13:16:55 2021-02-20T13:17:17.429 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T13:17:17.429 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T13:17:17.429 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:17:17.429 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror.mirror_status 2021-02-20T13:17:17.430 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T13:17:17.476 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T13:17:17.526 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local last_update state 2021-02-20T13:17:17.526 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T13:17:17.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ last_update=2021-02-20 13:16:55 2021-02-20T13:17:17.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T13:17:17.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ state=up+replaying 2021-02-20T13:17:17.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo up+replaying 2021-02-20T13:17:17.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep up+replaying 2021-02-20T13:17:17.539 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:up+replaying 2021-02-20T13:17:17.540 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:17:17.540 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:17:17.540 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap9 2021-02-20T13:17:17.540 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_snap cluster2 mirror test snap9 2021-02-20T13:17:17.541 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:17:17.541 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:17.541 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:17.542 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap9 2021-02-20T13:17:17.542 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap9 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:17:47.224 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T13:17:47.225 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:17:47.225 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:47.225 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:47.226 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T13:17:47.226 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:17:47.226 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:47.226 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:47.227 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state=started 2021-02-20T13:17:47.227 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:17:47.227 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:17:48.219 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9462615461268507) > weight (0.25) 2021-02-20T13:17:48.220 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8481008972847829) > weight (0.25) 2021-02-20T13:17:48.220 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.25986691234161086) > weight (0.25) 2021-02-20T13:17:48.220 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7200805486081117) > weight (0.25) 2021-02-20T13:17:48.221 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T13:17:48.232 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T13:17:48.234 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:17:48.234 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:48.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:48.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local test_state=started 2021-02-20T13:17:48.239 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_result 2021-02-20T13:17:48.240 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_state=stopped 2021-02-20T13:17:48.240 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T13:17:48.241 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_instance=cluster1 2021-02-20T13:17:48.241 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:17:48.241 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ + local cluster=cluster1 2021-02-20T13:17:48.242 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance=cluster1 2021-02-20T13:17:48.242 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:grep -i+ statelocal 2021-02-20T13:17:48.242 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: loop_instance 2021-02-20T13:17:48.243 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0 2021-02-20T13:17:48.243 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T13:17:48.244 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 0 3 2021-02-20T13:17:48.244 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T13:17:48.245 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:0 2021-02-20T13:17:48.245 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:17:48.245 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:17:48.246 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T13:17:48.246 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:17:48.252 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:17:48.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:17:48.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:17:48.254 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:17:48.255 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:17:48.255 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:17:48.255 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:17:48.256 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:17:48.259 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:17:48.267 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T13:17:48.267 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:0 2021-02-20T13:17:48.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:17:48.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:17:48.269 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T13:17:48.269 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:17:48.271 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:17:48.271 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:17:48.271 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:17:48.272 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:17:48.272 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:17:48.272 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:17:48.273 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:17:48.273 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:17:48.273 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:17:48.273 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T13:17:48.276 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:17:48.276 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:17:48.289 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:17:48.289 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T13:17:48.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:17:48.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:17:48.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:17:48.351 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T13:17:48.351 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:1 2021-02-20T13:17:48.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:17:48.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:17:48.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T13:17:48.353 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:17:48.353 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:17:48.355 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:17:48.355 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:17:48.356 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:17:48.356 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:17:48.357 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:17:48.357 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:17:48.357 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:17:48.358 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:17:48.358 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T13:17:48.359 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:1 2021-02-20T13:17:48.359 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:17:48.359 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:17:48.360 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T13:17:48.360 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:17:48.361 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:17:48.361 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:17:48.362 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:17:48.362 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:17:48.363 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:17:48.363 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:17:48.363 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:17:48.364 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:17:48.366 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:17:48.366 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T13:17:48.377 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:17:48.379 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:17:48.380 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:17:48.381 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T13:17:48.504 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:17:48.506 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:17:48.506 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:17:48.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T13:17:48.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:2 2021-02-20T13:17:48.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:17:48.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:17:48.509 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T13:17:48.509 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:17:48.510 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:17:48.510 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:17:48.511 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:17:48.511 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:17:48.511 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:17:48.512 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:17:48.512 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:17:48.513 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:17:48.513 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:17:48.513 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T13:17:48.514 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:2 2021-02-20T13:17:48.514 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:17:48.514 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:17:48.515 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T13:17:48.515 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:17:48.516 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:17:48.516 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:17:48.517 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:17:48.517 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:17:48.517 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:17:48.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:17:48.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:17:48.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:17:48.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:17:48.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T13:17:48.519 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:17:48.519 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:17:48.519 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:17:48.519 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T13:17:48.591 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:17:48.591 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:17:48.591 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:17:48.593 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T13:17:48.593 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:3 2021-02-20T13:17:48.593 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:17:48.594 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:17:48.594 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T13:17:48.594 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:17:48.594 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:17:48.594 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:17:48.595 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:17:48.596 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:17:48.597 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:17:48.597 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:17:48.597 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:17:48.597 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:17:48.597 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:17:48.598 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T13:17:48.598 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:3 2021-02-20T13:17:48.598 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:17:48.598 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:17:48.599 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T13:17:48.599 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:17:48.599 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:17:48.599 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:17:48.600 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:17:48.600 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:17:48.600 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:17:48.601 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:17:48.601 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:17:48.601 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:17:48.601 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:17:48.601 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T13:17:48.621 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:17:48.622 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:17:48.623 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:17:48.623 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T13:17:48.734 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:17:48.739 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ status_result= "state": "Replaying" 2021-02-20T13:17:48.747 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo "state": "Replaying" 2021-02-20T13:17:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep -i Replaying 2021-02-20T13:17:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_state=started 2021-02-20T13:17:48.758 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test started = started 2021-02-20T13:17:48.758 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:17:48.759 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T13:17:48.759 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:17:48.759 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:17:48.760 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:48.760 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:48.760 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = journal ] 2021-02-20T13:17:48.761 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = snapshot ] 2021-02-20T13:17:48.761 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T13:17:48.762 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:17:48.762 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:17:48.763 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:48.763 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:48.763 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T13:17:48.764 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ + echo cluster2-mirror-test.status 2021-02-20T13:17:48.764 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:sed -e s|[/ ]|_|g 2021-02-20T13:17:48.765 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:17:48.768 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T13:17:48.769 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror-test.status 2021-02-20T13:17:48.769 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:17:48.770 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: "state": "Replaying" 2021-02-20T13:17:48.771 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:17:48.772 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T13:17:48.772 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:17:48.772 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:48.773 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:48.773 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T13:17:49.244 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Snapshot ID: 38 2021-02-20T13:17:49.265 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:17:49.266 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:17:49.266 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:49.266 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:49.267 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:17:49.267 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T13:17:49.267 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:17:49.333 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T13:17:49.335 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snapshot_id=38 2021-02-20T13:17:49.335 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:17:49.336 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.2 2021-02-20T13:17:49.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:17:49.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:17:49.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:49.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:49.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:17:49.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:17:49.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:17:49.614 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:17:49.619 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id=34 2021-02-20T13:17:49.619 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 38 = 34 2021-02-20T13:17:49.620 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.4 2021-02-20T13:17:50.021 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:17:50.021 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:17:50.022 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:50.022 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:50.022 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:17:50.023 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:17:50.023 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:17:50.094 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:17:50.094 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:17:50.099 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:17:50.099 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:17:50.099 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:17:50.100 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:17:50.100 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 38 = 2021-02-20T13:17:50.100 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.8 2021-02-20T13:17:50.901 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:17:50.901 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:17:50.901 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:50.902 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:50.902 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:17:50.902 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:17:50.902 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:17:50.971 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:17:50.971 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:17:50.975 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:17:50.975 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:17:50.975 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:17:50.976 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:17:50.976 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 38 = 2021-02-20T13:17:50.976 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1.6 2021-02-20T13:17:52.577 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:17:52.577 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:17:52.578 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:52.578 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:52.578 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:17:52.578 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:17:52.578 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:17:52.646 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:17:52.646 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:17:52.649 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:17:52.649 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:17:52.650 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:17:52.650 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:17:52.650 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 38 = 2021-02-20T13:17:52.650 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 2 2021-02-20T13:17:54.651 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:17:54.651 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:17:54.651 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:54.652 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:54.652 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:17:54.652 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:17:54.652 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:17:54.819 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:17:54.819 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:17:54.820 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:17:54.821 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:17:54.821 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:17:54.821 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:17:54.821 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 38 = 2021-02-20T13:17:54.822 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 2 2021-02-20T13:17:56.823 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:17:56.823 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:17:56.824 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:56.824 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:56.824 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:17:56.824 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:17:56.825 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:17:56.926 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:17:56.933 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id=38 2021-02-20T13:17:56.933 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 38 = 38 2021-02-20T13:17:56.934 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:17:56.934 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snap_present cluster1 mirror test snap9 2021-02-20T13:17:56.934 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:17:56.934 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:56.935 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:56.935 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap9 2021-02-20T13:17:56.935 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:17:56.935 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:17:57.933 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 info mirror/test@snap9 2021-02-20T13:17:58.006 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:rbd image 'test': 2021-02-20T13:17:58.006 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: size 512 MiB in 128 objects 2021-02-20T13:17:58.006 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: order 22 (4 MiB objects) 2021-02-20T13:17:58.006 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: snapshot_count: 18 2021-02-20T13:17:58.007 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: id: 109d36d23f3 2021-02-20T13:17:58.007 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: block_name_prefix: rbd_data.109d36d23f3 2021-02-20T13:17:58.007 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: format: 2 2021-02-20T13:17:58.007 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: features: layering, exclusive-lock, non-primary 2021-02-20T13:17:58.008 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: op_features: 2021-02-20T13:17:58.008 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: flags: 2021-02-20T13:17:58.008 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: create_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:17:58.008 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: access_timestamp: Sat Feb 20 13:17:03 2021 2021-02-20T13:17:58.008 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: modify_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:17:58.009 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: protected: False 2021-02-20T13:17:58.009 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring state: enabled 2021-02-20T13:17:58.009 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring mode: snapshot 2021-02-20T13:17:58.009 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring global id: 49a0b5eb-7d69-4824-806b-1ece12354eb9 2021-02-20T13:17:58.010 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring primary: false 2021-02-20T13:17:58.011 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:17:58.011 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snap8-clean ] 2021-02-20T13:17:58.011 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap8-clean 2021-02-20T13:17:58.012 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:17:58.012 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:17:58.012 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap8-clean 2021-02-20T13:17:58.012 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:17:58.013 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:17:58.013 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:17:58.013 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:17:58.014 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8-clean /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:17:58.858 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:17:58.863 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8-clean /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:17:59.523 INFO:tasks.ceph.cluster1.osd.1.smithi170.stderr:2021-02-20T13:17:59.516+0000 7fb0f14b2700 -1 bdev(0x559d40872000 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 1 2021-02-20T13:17:59.875 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:17:59.878 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:18:00.156 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:18:00.284 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:18:00.284 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap9 2021-02-20T13:18:00.285 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:00.285 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:00.285 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap9 2021-02-20T13:18:00.285 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:18:00.286 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:18:00.287 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:18:00.287 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:18:00.288 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9 /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:18:01.072 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:18:01.079 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9 /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:18:02.086 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:18:02.091 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:18:02.251 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4921956598407766) > weight (0.25) 2021-02-20T13:18:02.252 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.29227883887771056) > weight (0.25) 2021-02-20T13:18:02.252 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.765593927271113) > weight (0.25) 2021-02-20T13:18:02.252 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5375148398806264) > weight (0.25) 2021-02-20T13:18:02.252 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T13:18:02.377 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:18:02.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:18:02.506 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ clean_snap_name=snap9-clean 2021-02-20T13:18:02.506 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_snap cluster2 mirror test snap9-clean 2021-02-20T13:18:02.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:18:02.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:02.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:02.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap9-clean 2021-02-20T13:18:02.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap9-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T13:18:03.310 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T13:18:03.310 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:18:03.311 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:03.311 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:03.311 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:18:03.312 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=7 2021-02-20T13:18:03.313 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:18:03.313 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T13:18:04.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 44059648 bytes 2021-02-20T13:18:05.048 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 73704448 bytes 2021-02-20T13:18:05.726 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 114339840 bytes 2021-02-20T13:18:06.104 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 134203392 bytes 2021-02-20T13:18:06.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 194702336 bytes 2021-02-20T13:18:07.173 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 255847424 bytes 2021-02-20T13:18:07.773 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 323481600 bytes 2021-02-20T13:18:08.046 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 363180032 bytes 2021-02-20T13:18:09.088 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 461008896 bytes 2021-02-20T13:18:09.230 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 461612032 bytes 2021-02-20T13:18:09.568 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 496861184 bytes 2021-02-20T13:18:09.991 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 559604736 bytes 2021-02-20T13:18:10.315 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:18:10.316 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:18:10.316 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:18:10.316 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:18:10.316 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T13:18:10.317 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:10.317 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:10.317 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:10.317 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:18:10.317 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local description_pattern= 2021-02-20T13:18:10.318 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local service_pattern= 2021-02-20T13:18:10.318 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:18:11.318 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T13:18:11.318 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:11.318 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:11.318 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:11.319 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:18:11.319 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local description_pattern= 2021-02-20T13:18:11.319 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local service_pattern= 2021-02-20T13:18:11.319 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T13:18:11.320 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T13:18:11.320 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:18:11.321 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:18:11.321 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T13:18:11.321 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ tee /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:18:11.390 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:test: 2021-02-20T13:18:11.391 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: global_id: 49a0b5eb-7d69-4824-806b-1ece12354eb9 2021-02-20T13:18:11.391 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state: up+replaying 2021-02-20T13:18:11.391 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: description: replaying, {"bytes_per_second":577297.0699999999,"bytes_per_snapshot":403611648.0,"local_snapshot_timestamp":1613827013,"remote_snapshot_timestamp":1613827069,"replay_state":"syncing","seconds_until_synced":699,"syncing_percent":13,"syncing_snapshot_timestamp":1613827069} 2021-02-20T13:18:11.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: service: mirror.3 on smithi183 2021-02-20T13:18:11.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: last_update: 2021-02-20 13:17:55 2021-02-20T13:18:11.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: peer_sites: 2021-02-20T13:18:11.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: name: cluster2 2021-02-20T13:18:11.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: state: up+stopped 2021-02-20T13:18:11.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: description: local image is primary 2021-02-20T13:18:11.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: last_update: 2021-02-20 13:17:55 2021-02-20T13:18:11.402 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:18:11.403 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: state: up+replaying 2021-02-20T13:18:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^ description: .* /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:18:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: description: replaying, {"bytes_per_second":577297.0699999999,"bytes_per_snapshot":403611648.0,"local_snapshot_timestamp":1613827013,"remote_snapshot_timestamp":1613827069,"replay_state":"syncing","seconds_until_synced":699,"syncing_percent":13,"syncing_snapshot_timestamp":1613827069} 2021-02-20T13:18:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:18:11.407 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo up+replaying 2021-02-20T13:18:11.407 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep ^up+ 2021-02-20T13:18:11.409 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:up+replaying 2021-02-20T13:18:11.409 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: service: mirror.3 on smithi183 2021-02-20T13:18:11.410 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep service: *mirror..* on /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:18:11.410 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.mirror_status 2021-02-20T13:18:11.413 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local last_update=2021-02-20 13:17:55 2021-02-20T13:18:11.415 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 13:17:55 2021-02-20T13:18:11.415 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:11.416 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:11.416 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:11.416 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state_pattern=up+replaying 2021-02-20T13:18:11.417 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local prev_last_update=2021-02-20 13:17:55 2021-02-20T13:18:11.417 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T13:18:11.417 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T13:18:11.418 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:18:11.418 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror.mirror_status 2021-02-20T13:18:11.418 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T13:18:11.453 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T13:18:11.511 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local last_update state 2021-02-20T13:18:11.511 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T13:18:11.516 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ last_update=2021-02-20 13:17:55 2021-02-20T13:18:11.516 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T13:18:11.522 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ state=up+replaying 2021-02-20T13:18:11.523 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo up+replaying 2021-02-20T13:18:11.523 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep up+replaying 2021-02-20T13:18:11.534 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:up+replaying 2021-02-20T13:18:11.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:18:11.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:18:11.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap10 2021-02-20T13:18:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_snap cluster2 mirror test snap10 2021-02-20T13:18:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:18:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:11.539 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap10 2021-02-20T13:18:11.539 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap10 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:18:40.343 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T13:18:40.344 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:40.344 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:40.344 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:40.344 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T13:18:40.345 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:40.345 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:40.345 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:40.345 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state=started 2021-02-20T13:18:40.345 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:18:40.346 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:18:41.344 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T13:18:41.344 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:41.345 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:41.345 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:41.345 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local test_state=started 2021-02-20T13:18:41.345 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_result 2021-02-20T13:18:41.345 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_state=stopped 2021-02-20T13:18:41.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T13:18:41.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_instance=cluster1 2021-02-20T13:18:41.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ + shift 2021-02-20T13:18:41.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ localgrep cluster=cluster1 2021-02-20T13:18:41.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: -i state 2021-02-20T13:18:41.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance=cluster1 2021-02-20T13:18:41.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loop_instance 2021-02-20T13:18:41.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0 2021-02-20T13:18:41.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T13:18:41.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 0 3 2021-02-20T13:18:41.350 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T13:18:41.351 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:0 2021-02-20T13:18:41.351 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:18:41.351 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:18:41.351 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T13:18:41.351 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:18:41.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:18:41.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:18:41.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:41.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:18:41.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:18:41.353 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:18:41.353 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:41.353 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:18:41.353 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:18:41.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T13:18:41.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:0 2021-02-20T13:18:41.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:41.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:18:41.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T13:18:41.355 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:18:41.355 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:18:41.355 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:18:41.355 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:41.355 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:18:41.356 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:18:41.356 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:18:41.356 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:18:41.356 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:18:41.356 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:18:41.357 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T13:18:41.371 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:18:41.371 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:18:41.372 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:18:41.372 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T13:18:41.478 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:18:41.478 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:18:41.478 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:18:41.480 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T13:18:41.481 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:1 2021-02-20T13:18:41.481 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:18:41.481 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:18:41.481 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T13:18:41.481 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:18:41.482 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:18:41.482 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:18:41.482 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:41.482 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:18:41.483 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:18:41.484 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:18:41.484 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:41.484 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:18:41.484 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:18:41.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T13:18:41.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:1 2021-02-20T13:18:41.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:41.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:18:41.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T13:18:41.486 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:18:41.486 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:18:41.486 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:18:41.486 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:41.486 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:18:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:18:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:18:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:18:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:18:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:18:41.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T13:18:41.502 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:18:41.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:18:41.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:18:41.504 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T13:18:41.600 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:18:41.602 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:18:41.603 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:18:41.607 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T13:18:41.607 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:2 2021-02-20T13:18:41.608 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:18:41.608 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:18:41.609 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T13:18:41.609 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:18:41.609 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:18:41.610 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:18:41.610 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:41.611 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:18:41.611 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:18:41.612 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:18:41.612 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:41.612 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:18:41.613 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:18:41.613 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T13:18:41.614 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:2 2021-02-20T13:18:41.614 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:41.615 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:18:41.615 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T13:18:41.616 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:18:41.616 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:18:41.617 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:18:41.617 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:41.617 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:18:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:18:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:18:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:18:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:18:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:18:41.620 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T13:18:41.622 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:18:41.622 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:18:41.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:18:41.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T13:18:41.716 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:18:41.716 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:18:41.717 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:18:41.717 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T13:18:41.718 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:3 2021-02-20T13:18:41.719 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:18:41.719 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:18:41.720 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T13:18:41.720 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:18:41.721 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:18:41.721 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:18:41.721 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:41.722 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:18:41.722 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:18:41.723 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:18:41.724 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:41.724 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:18:41.724 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:18:41.725 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T13:18:41.725 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:3 2021-02-20T13:18:41.725 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:41.725 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:18:41.725 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T13:18:41.726 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:18:41.726 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:18:41.726 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:18:41.726 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:41.727 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:18:41.727 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:18:41.727 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:18:41.727 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:18:41.727 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:18:41.728 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:18:41.728 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T13:18:41.737 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:18:41.738 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:18:41.738 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:18:41.738 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T13:18:41.843 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:18:41.844 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ status_result= "state": "Replaying" 2021-02-20T13:18:41.844 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ + grepecho -i Replaying "state": "Replaying" 2021-02-20T13:18:41.844 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:18:41.847 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: "state": "Replaying" 2021-02-20T13:18:41.847 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_state=started 2021-02-20T13:18:41.847 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test started = started 2021-02-20T13:18:41.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:18:41.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T13:18:41.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:18:41.849 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:18:41.849 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:41.849 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:41.849 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = journal ] 2021-02-20T13:18:41.849 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = snapshot ] 2021-02-20T13:18:41.850 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T13:18:41.850 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:18:41.850 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:18:41.850 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:41.851 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:41.851 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T13:18:41.851 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster2-mirror-test.status 2021-02-20T13:18:41.851 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:18:41.852 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:18:41.852 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T13:18:41.852 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror-test.status 2021-02-20T13:18:41.853 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:18:41.854 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:18:41.854 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T13:18:41.855 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:18:41.855 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:41.856 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:41.856 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T13:18:42.369 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Snapshot ID: 42 2021-02-20T13:18:42.391 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:18:42.391 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:18:42.391 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:42.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:42.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:18:42.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T13:18:42.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:18:42.448 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T13:18:42.452 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snapshot_id=42 2021-02-20T13:18:42.452 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:18:42.453 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.2 2021-02-20T13:18:42.654 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:18:42.654 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:42.655 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:42.655 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:42.655 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:18:42.656 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:18:42.656 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:18:42.728 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:18:42.732 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id=38 2021-02-20T13:18:42.733 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 42 = 38 2021-02-20T13:18:42.733 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.4 2021-02-20T13:18:43.134 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:18:43.134 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:43.135 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:43.135 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:43.135 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:18:43.135 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:18:43.135 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:18:43.212 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:18:43.213 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:18:43.217 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:18:43.217 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:18:43.218 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:18:43.218 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:18:43.219 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 42 = 2021-02-20T13:18:43.219 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.8 2021-02-20T13:18:44.020 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:18:44.020 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:44.020 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:44.021 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:44.021 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:18:44.021 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:18:44.021 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:18:44.098 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:18:44.098 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:18:44.102 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:18:44.103 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:18:44.103 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:18:44.103 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:18:44.104 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 42 = 2021-02-20T13:18:44.104 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1.6 2021-02-20T13:18:45.704 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:18:45.705 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:45.705 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:45.705 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:45.705 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:18:45.705 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:18:45.706 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:18:45.766 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:18:45.766 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:18:45.771 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:18:45.771 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:18:45.771 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:18:45.772 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:18:45.772 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 42 = 2021-02-20T13:18:45.772 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 2 2021-02-20T13:18:47.773 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:18:47.774 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:47.774 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:47.774 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:47.774 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:18:47.775 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:18:47.775 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:18:47.845 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:18:47.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id=42 2021-02-20T13:18:47.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 42 = 42 2021-02-20T13:18:47.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:18:47.849 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snap_present cluster1 mirror test snap10 2021-02-20T13:18:47.849 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:47.849 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:47.850 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:47.850 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap10 2021-02-20T13:18:47.850 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:18:47.850 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:18:48.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 info mirror/test@snap10 2021-02-20T13:18:48.925 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:rbd image 'test': 2021-02-20T13:18:48.926 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: size 512 MiB in 128 objects 2021-02-20T13:18:48.926 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: order 22 (4 MiB objects) 2021-02-20T13:18:48.926 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: snapshot_count: 21 2021-02-20T13:18:48.926 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: id: 109d36d23f3 2021-02-20T13:18:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: block_name_prefix: rbd_data.109d36d23f3 2021-02-20T13:18:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: format: 2 2021-02-20T13:18:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: features: layering, exclusive-lock, operations, non-primary 2021-02-20T13:18:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: op_features: snap-trash 2021-02-20T13:18:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: flags: 2021-02-20T13:18:48.928 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: create_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:18:48.928 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: access_timestamp: Sat Feb 20 13:17:03 2021 2021-02-20T13:18:48.928 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: modify_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:18:48.928 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: protected: False 2021-02-20T13:18:48.928 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring state: enabled 2021-02-20T13:18:48.929 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring mode: snapshot 2021-02-20T13:18:48.929 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring global id: 49a0b5eb-7d69-4824-806b-1ece12354eb9 2021-02-20T13:18:48.929 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring primary: false 2021-02-20T13:18:48.932 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:18:48.932 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snap9-clean ] 2021-02-20T13:18:48.932 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap9-clean 2021-02-20T13:18:48.932 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:48.933 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:48.933 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap9-clean 2021-02-20T13:18:48.933 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:18:48.933 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:18:48.933 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:18:48.934 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:18:48.934 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9-clean /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:18:49.768 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:18:49.772 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9-clean /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:18:50.847 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:18:50.851 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:18:51.127 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:18:51.252 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:18:51.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap10 2021-02-20T13:18:51.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:51.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:51.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap10 2021-02-20T13:18:51.254 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:18:51.254 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:18:51.254 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:18:51.254 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:18:51.255 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap10 /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:18:52.107 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:18:52.110 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap10 /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:18:53.100 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:18:53.106 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:18:53.389 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:18:53.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:18:53.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ clean_snap_name=snap10-clean 2021-02-20T13:18:53.509 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_snap cluster2 mirror test snap10-clean 2021-02-20T13:18:53.509 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:18:53.510 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:53.510 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:53.510 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap10-clean 2021-02-20T13:18:53.511 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap10-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T13:18:54.346 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T13:18:54.346 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:54.346 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:54.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:54.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T13:18:54.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:54.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:54.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:54.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state=started 2021-02-20T13:18:54.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:18:54.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:18:55.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T13:18:55.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:55.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:55.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:55.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local test_state=started 2021-02-20T13:18:55.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_result 2021-02-20T13:18:55.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_state=stopped 2021-02-20T13:18:55.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T13:18:55.350 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ + greplocal -i cluster_instance=cluster1 state 2021-02-20T13:18:55.350 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:18:55.350 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:18:55.350 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:55.351 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance=cluster1 2021-02-20T13:18:55.351 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loop_instance 2021-02-20T13:18:55.351 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0 2021-02-20T13:18:55.351 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T13:18:55.351 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 0 3 2021-02-20T13:18:55.353 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T13:18:55.353 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:0 2021-02-20T13:18:55.353 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:18:55.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:18:55.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T13:18:55.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:18:55.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:18:55.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:18:55.355 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:55.355 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:18:55.355 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:18:55.355 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:18:55.356 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:55.356 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:18:55.356 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:18:55.356 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T13:18:55.356 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:0 2021-02-20T13:18:55.357 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:55.357 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:18:55.357 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T13:18:55.357 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:0 2021-02-20T13:18:55.357 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:18:55.358 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:18:55.358 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:55.358 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:18:55.358 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T13:18:55.359 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:18:55.359 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:18:55.359 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=0 2021-02-20T13:18:55.360 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=0 2021-02-20T13:18:55.360 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T13:18:55.373 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:18:55.374 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:18:55.374 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T13:18:55.374 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T13:18:55.482 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:18:55.482 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:18:55.483 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:18:55.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T13:18:55.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:1 2021-02-20T13:18:55.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:18:55.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:18:55.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T13:18:55.486 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:18:55.486 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:18:55.486 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:18:55.486 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:55.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:18:55.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:18:55.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:18:55.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:55.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:18:55.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:18:55.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T13:18:55.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:1 2021-02-20T13:18:55.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:55.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:18:55.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T13:18:55.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:1 2021-02-20T13:18:55.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:18:55.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:18:55.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:55.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:18:55.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T13:18:55.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:18:55.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:18:55.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=1 2021-02-20T13:18:55.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=1 2021-02-20T13:18:55.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T13:18:55.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:18:55.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:18:55.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T13:18:55.506 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T13:18:55.603 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:18:55.604 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:18:55.604 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:18:55.605 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T13:18:55.606 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:2 2021-02-20T13:18:55.606 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:18:55.606 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:18:55.606 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T13:18:55.607 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:18:55.607 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:18:55.607 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:18:55.607 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:55.608 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:18:55.608 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:18:55.608 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:18:55.608 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:55.609 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:18:55.609 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:18:55.609 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T13:18:55.609 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:2 2021-02-20T13:18:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:18:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T13:18:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:2 2021-02-20T13:18:55.611 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:18:55.611 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:18:55.611 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:55.611 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:18:55.612 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T13:18:55.612 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:18:55.613 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:18:55.613 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=2 2021-02-20T13:18:55.613 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=2 2021-02-20T13:18:55.613 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T13:18:55.623 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:18:55.623 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:18:55.623 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T13:18:55.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T13:18:55.721 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T13:18:55.721 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T13:18:55.722 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:admin_socket: invalid command 2021-02-20T13:18:55.723 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T13:18:55.723 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1:3 2021-02-20T13:18:55.724 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:18:55.724 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:18:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T13:18:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:18:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=cluster 2021-02-20T13:18:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:18:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:18:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:18:55.727 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval cluster=cluster1 2021-02-20T13:18:55.727 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:55.727 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:18:55.727 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:18:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T13:18:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1:3 2021-02-20T13:18:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance 2021-02-20T13:18:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T13:18:55.729 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local val=cluster1:3 2021-02-20T13:18:55.729 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster_var_name=local_cluster 2021-02-20T13:18:55.730 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local instance_var_name=instance 2021-02-20T13:18:55.730 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cluster=cluster1 2021-02-20T13:18:55.730 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:18:55.731 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T13:18:55.731 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval local_cluster=cluster1 2021-02-20T13:18:55.731 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local_cluster=cluster1 2021-02-20T13:18:55.732 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ eval instance=3 2021-02-20T13:18:55.732 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ instance=3 2021-02-20T13:18:55.732 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T13:18:55.744 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:18:55.745 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:18:55.745 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T13:18:55.745 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T13:18:55.850 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:18:55.851 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ status_result= "state": "Replaying" 2021-02-20T13:18:55.852 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo "state": "Replaying" 2021-02-20T13:18:55.852 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ grep -i Replaying 2021-02-20T13:18:55.854 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: "state": "Replaying" 2021-02-20T13:18:55.855 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_state=started 2021-02-20T13:18:55.855 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test started = started 2021-02-20T13:18:55.855 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:18:55.856 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T13:18:55.856 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:18:55.856 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:18:55.857 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:55.857 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:55.857 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = journal ] 2021-02-20T13:18:55.857 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ snapshot = snapshot ] 2021-02-20T13:18:55.857 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T13:18:55.858 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_cluster=cluster1 2021-02-20T13:18:55.858 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:18:55.858 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:55.858 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:55.858 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T13:18:55.859 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster2-mirror-test.status 2021-02-20T13:18:55.859 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:18:55.860 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local status_log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:18:55.860 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T13:18:55.861 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo cluster1-mirror-test.status 2021-02-20T13:18:55.861 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T13:18:55.861 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local local_status_log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:18:55.864 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T13:18:55.864 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:18:55.865 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:55.865 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:55.865 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T13:18:56.374 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Snapshot ID: 45 2021-02-20T13:18:56.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:18:56.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:18:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.status 2021-02-20T13:18:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T13:18:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:18:56.456 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T13:18:56.459 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snapshot_id=45 2021-02-20T13:18:56.459 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:18:56.460 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.2 2021-02-20T13:18:56.660 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:18:56.660 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:56.661 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:56.661 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:56.661 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:18:56.661 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:18:56.662 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:18:56.734 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:18:56.740 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id=42 2021-02-20T13:18:56.740 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 45 = 42 2021-02-20T13:18:56.740 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.4 2021-02-20T13:18:57.141 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:18:57.142 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:57.142 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:57.142 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:57.142 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:18:57.143 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:18:57.143 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:18:57.215 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ true 2021-02-20T13:18:57.215 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:18:57.219 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:-:1.1: Document is empty 2021-02-20T13:18:57.219 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: 2021-02-20T13:18:57.220 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:^ 2021-02-20T13:18:57.220 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id= 2021-02-20T13:18:57.221 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 45 = 2021-02-20T13:18:57.221 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.8 2021-02-20T13:18:58.022 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:18:58.022 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:58.022 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:58.023 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:58.023 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local log=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.status 2021-02-20T13:18:58.023 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T13:18:58.023 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T13:18:58.094 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T13:18:58.100 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local primary_snapshot_id=45 2021-02-20T13:18:58.100 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test 45 = 45 2021-02-20T13:18:58.100 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:18:58.100 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_snap_present cluster1 mirror test snap10-clean 2021-02-20T13:18:58.101 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:18:58.101 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:58.101 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:58.101 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap10-clean 2021-02-20T13:18:58.101 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:18:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:18:59.101 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 info mirror/test@snap10-clean 2021-02-20T13:18:59.171 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:rbd image 'test': 2021-02-20T13:18:59.171 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: size 512 MiB in 128 objects 2021-02-20T13:18:59.171 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: order 22 (4 MiB objects) 2021-02-20T13:18:59.172 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: snapshot_count: 21 2021-02-20T13:18:59.172 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: id: 109d36d23f3 2021-02-20T13:18:59.172 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: block_name_prefix: rbd_data.109d36d23f3 2021-02-20T13:18:59.172 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: format: 2 2021-02-20T13:18:59.172 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: features: layering, exclusive-lock, non-primary 2021-02-20T13:18:59.173 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: op_features: 2021-02-20T13:18:59.173 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: flags: 2021-02-20T13:18:59.173 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: create_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:18:59.173 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: access_timestamp: Sat Feb 20 13:18:49 2021 2021-02-20T13:18:59.173 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: modify_timestamp: Sat Feb 20 13:09:56 2021 2021-02-20T13:18:59.174 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: protected: False 2021-02-20T13:18:59.174 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring state: enabled 2021-02-20T13:18:59.174 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring mode: snapshot 2021-02-20T13:18:59.174 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring global id: 49a0b5eb-7d69-4824-806b-1ece12354eb9 2021-02-20T13:18:59.175 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout: mirroring primary: false 2021-02-20T13:18:59.176 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:18:59.177 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 1 10 2021-02-20T13:18:59.178 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap1 2021-02-20T13:18:59.178 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap1 2021-02-20T13:18:59.178 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:18:59.179 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:18:59.179 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap1 2021-02-20T13:18:59.179 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:18:59.179 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:18:59.179 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:18:59.180 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:18:59.180 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1 /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:18:59.997 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:00.001 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1 /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:00.278 INFO:tasks.ceph.cluster1.osd.1.smithi170.stderr:2021-02-20T13:19:00.266+0000 7fb0f04b0700 -1 bdev(0x559d40872000 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 1 2021-02-20T13:19:00.278 INFO:tasks.ceph.cluster1.osd.1.smithi170.stderr:2021-02-20T13:19:00.270+0000 7fb0f34b6700 -1 bdev(0x559d40872000 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 1 2021-02-20T13:19:00.279 INFO:tasks.ceph.cluster1.osd.1.smithi170.stderr:2021-02-20T13:19:00.270+0000 7fb0f14b2700 -1 bdev(0x559d40872000 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 1 2021-02-20T13:19:00.279 INFO:tasks.ceph.cluster1.osd.1.smithi170.stderr:2021-02-20T13:19:00.270+0000 7fb0f3cb7700 -1 bdev(0x559d40872000 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 1 2021-02-20T13:19:00.914 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:00.919 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:01.196 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:01.321 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:19:01.321 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap1-clean 2021-02-20T13:19:01.322 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap1-clean 2021-02-20T13:19:01.322 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:01.322 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:01.322 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap1-clean 2021-02-20T13:19:01.323 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:19:01.323 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:01.323 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:01.323 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:01.324 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1-clean /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:02.125 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:02.129 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1-clean /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:03.045 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:03.052 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:03.331 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:03.458 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:19:03.458 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap2 2021-02-20T13:19:03.459 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap2 2021-02-20T13:19:03.459 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:03.459 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:03.459 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap2 2021-02-20T13:19:03.460 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:19:03.460 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:03.460 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:03.460 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:03.461 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2 /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:04.288 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:04.292 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2 /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:04.571 INFO:tasks.ceph.cluster1.osd.0.smithi170.stderr:2021-02-20T13:19:04.562+0000 7f183c53a700 -1 bdev(0x5654f55f4000 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2021-02-20T13:19:04.572 INFO:tasks.ceph.cluster1.osd.0.smithi170.stderr:2021-02-20T13:19:04.562+0000 7f183ed3f700 -1 bdev(0x5654f55f4000 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2021-02-20T13:19:05.340 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:05.344 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:05.628 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:05.755 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:19:05.755 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap2-clean 2021-02-20T13:19:05.756 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap2-clean 2021-02-20T13:19:05.756 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:05.756 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:05.756 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap2-clean 2021-02-20T13:19:05.756 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:19:05.757 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:05.757 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:05.757 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:05.758 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2-clean /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:06.550 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:06.554 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2-clean /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:07.475 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:07.479 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:07.758 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:07.886 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:19:07.887 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap3 2021-02-20T13:19:07.887 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap3 2021-02-20T13:19:07.887 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:07.887 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:07.887 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap3 2021-02-20T13:19:07.888 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:19:07.888 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:07.888 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:07.888 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:07.889 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3 /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:08.756 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:08.762 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3 /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:09.800 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:09.804 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:10.081 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:10.208 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:19:10.209 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap3-clean 2021-02-20T13:19:10.209 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap3-clean 2021-02-20T13:19:10.209 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:10.209 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:10.210 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap3-clean 2021-02-20T13:19:10.210 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:19:10.210 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:10.210 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:10.211 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:10.211 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3-clean /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:11.050 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3-clean /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:12.075 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:12.079 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:12.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:19:12.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap4 2021-02-20T13:19:12.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap4 2021-02-20T13:19:12.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:12.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:12.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap4 2021-02-20T13:19:12.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:19:12.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:12.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:12.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:12.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4 /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:13.310 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:13.314 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4 /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:13.654 INFO:tasks.ceph.cluster1.osd.0.smithi170.stderr:2021-02-20T13:19:13.646+0000 7f1839d35700 -1 bdev(0x5654f55f4000 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 2 2021-02-20T13:19:13.654 INFO:tasks.ceph.cluster1.osd.0.smithi170.stderr:2021-02-20T13:19:13.646+0000 7f183ed3f700 -1 bdev(0x5654f55f4000 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 2 2021-02-20T13:19:14.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:14.398 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:14.679 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:14.807 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:19:14.807 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap4-clean 2021-02-20T13:19:14.808 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap4-clean 2021-02-20T13:19:14.808 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:14.808 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:14.808 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap4-clean 2021-02-20T13:19:14.809 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:19:14.809 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:14.809 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:14.809 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:14.810 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4-clean /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:15.686 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:15.690 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4-clean /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:16.172 INFO:tasks.ceph.cluster1.osd.0.smithi170.stderr:2021-02-20T13:19:16.166+0000 7f183f540700 -1 bdev(0x5654f55f4000 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2021-02-20T13:19:16.671 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:16.676 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:16.960 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:17.127 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:19:17.127 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap5 2021-02-20T13:19:17.128 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap5 2021-02-20T13:19:17.128 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:17.128 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:17.128 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap5 2021-02-20T13:19:17.128 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:19:17.129 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:17.129 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:17.129 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:17.130 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5 /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:17.947 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:17.950 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5 /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:19.005 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:19.009 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:19.289 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:19.411 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:19:19.412 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap5-clean 2021-02-20T13:19:19.412 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap5-clean 2021-02-20T13:19:19.412 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:19.412 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:19.412 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap5-clean 2021-02-20T13:19:19.413 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:19:19.413 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:19.413 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:19.413 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:19.414 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5-clean /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:20.243 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:20.247 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5-clean /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:21.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:21.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:21.773 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:21.893 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:19:21.893 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap6 2021-02-20T13:19:21.893 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap6 2021-02-20T13:19:21.893 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:21.894 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:21.894 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap6 2021-02-20T13:19:21.894 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:19:21.894 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:21.895 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:21.895 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:21.895 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6 /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:21.982 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T13:19:21.973+0000 7f4a0bb52700 -1 bdev(0x55fc57f52000 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 1 2021-02-20T13:19:22.725 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:22.729 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6 /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:23.686 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:23.690 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:23.969 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:24.097 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:19:24.098 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap6-clean 2021-02-20T13:19:24.098 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap6-clean 2021-02-20T13:19:24.098 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:24.099 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:24.099 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap6-clean 2021-02-20T13:19:24.099 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:19:24.099 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:24.099 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:24.100 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:24.100 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6-clean /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:24.222 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:19:24.223 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:19:24.224 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T13:19:24.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T13:19:24.221+0000 7f92d6118700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 19163) UID: 1000 2021-02-20T13:19:24.253 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8139998557217162) > weight (0.25) 2021-02-20T13:19:24.254 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5803653888642645) > weight (0.25) 2021-02-20T13:19:24.254 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T13:19:24.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T13:19:24.255 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T13:19:24.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T13:19:24.253+0000 7f0af4a0f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 24662) UID: 1000 2021-02-20T13:19:24.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:19:24.625 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:19:24.975 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:24.980 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6-clean /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:26.121 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:26.124 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:26.402 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:26.530 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:19:26.531 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap7 2021-02-20T13:19:26.531 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap7 2021-02-20T13:19:26.531 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:26.531 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:26.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap7 2021-02-20T13:19:26.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:19:26.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:26.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:26.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:26.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7 /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:27.256 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T13:19:27.256 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:19:27.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:19:27.257 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T13:19:27.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T13:19:27.258 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T13:19:27.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T13:19:27.260 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T13:19:27.373 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:27.378 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7 /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:28.414 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:28.417 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:28.699 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:28.827 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:19:28.827 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap7-clean 2021-02-20T13:19:28.827 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap7-clean 2021-02-20T13:19:28.828 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:28.828 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:28.828 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap7-clean 2021-02-20T13:19:28.828 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:19:28.828 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:28.829 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:28.829 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:28.829 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7-clean /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:29.663 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:29.667 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7-clean /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:30.722 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:30.727 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:31.003 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:31.132 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:19:31.132 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap8 2021-02-20T13:19:31.132 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap8 2021-02-20T13:19:31.132 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:31.133 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:31.133 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap8 2021-02-20T13:19:31.133 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:19:31.133 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:31.134 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:31.134 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:31.134 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8 /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:31.225 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:19:31.225 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:19:31.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:19:31.226 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:19:31.226 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:19:31.227 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:19:31.230 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:19:31.230 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T13:19:32.035 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:32.038 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8 /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:33.143 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:33.146 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:33.426 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:33.555 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:19:33.556 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap8-clean 2021-02-20T13:19:33.556 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap8-clean 2021-02-20T13:19:33.556 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:33.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:33.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap8-clean 2021-02-20T13:19:33.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:19:33.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:33.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:33.558 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:33.558 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8-clean /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:33.838 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T13:19:33.836+0000 7f03c051f700 -1 bdev(0x556b6bcb8000 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 1 2021-02-20T13:19:33.839 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T13:19:33.836+0000 7f03c1d22700 -1 bdev(0x556b6bcb8000 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 1 2021-02-20T13:19:34.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:34.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8-clean /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:35.423 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:35.427 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:35.711 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:35.840 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:19:35.840 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap9 2021-02-20T13:19:35.840 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap9 2021-02-20T13:19:35.841 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:35.841 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:35.841 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap9 2021-02-20T13:19:35.841 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:19:35.841 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:35.842 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:35.842 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:35.842 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9 /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:36.660 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:36.664 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9 /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:37.617 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:37.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:37.909 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:38.036 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:19:38.036 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap9-clean 2021-02-20T13:19:38.036 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap9-clean 2021-02-20T13:19:38.036 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:38.037 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:38.037 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap9-clean 2021-02-20T13:19:38.037 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:19:38.037 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:38.038 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:38.038 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:38.038 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9-clean /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:38.897 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:38.901 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9-clean /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:39.787 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:39.793 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:40.070 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:40.199 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:19:40.200 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap10 2021-02-20T13:19:40.200 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap10 2021-02-20T13:19:40.200 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:40.200 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:40.200 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap10 2021-02-20T13:19:40.201 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:19:40.201 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:40.201 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:40.201 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:40.202 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap10 /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:41.044 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:41.049 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap10 /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:42.022 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:42.026 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:42.310 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:42.436 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:19:42.437 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap10-clean 2021-02-20T13:19:42.437 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ compare_image_snaps mirror test snap10-clean 2021-02-20T13:19:42.437 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:42.438 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:42.438 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap_name=snap10-clean 2021-02-20T13:19:42.438 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local ret=0 2021-02-20T13:19:42.438 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local rmt_export=/tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:42.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local loc_export=/tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:42.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:42.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 -p mirror export test@snap10-clean /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export 2021-02-20T13:19:43.240 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:43.243 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster1 -p mirror export test@snap10-clean /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:44.297 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T13:19:44.303 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ cmp /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:44.583 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rm -f /tmp/tmp.CsCtSCD9Zs/cluster2-mirror-test.export /tmp/tmp.CsCtSCD9Zs/cluster1-mirror-test.export 2021-02-20T13:19:44.710 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:19:44.711 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 1 10 2021-02-20T13:19:44.711 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap1 2021-02-20T13:19:44.712 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_snapshot cluster2 mirror test snap1 2021-02-20T13:19:44.712 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:19:44.712 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:44.712 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:44.712 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap=snap1 2021-02-20T13:19:44.713 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap1 2021-02-20T13:19:45.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Removing snap: 100% complete...done. 2021-02-20T13:19:45.365 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap1-clean 2021-02-20T13:19:45.365 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_snapshot cluster2 mirror test snap1-clean 2021-02-20T13:19:45.366 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:19:45.366 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:45.366 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:45.367 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap=snap1-clean 2021-02-20T13:19:45.367 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap1-clean 2021-02-20T13:19:46.376 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Removing snap: 100% complete...done. 2021-02-20T13:19:46.432 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap2 2021-02-20T13:19:46.432 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_snapshot cluster2 mirror test snap2 2021-02-20T13:19:46.433 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:19:46.433 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:46.433 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:46.433 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap=snap2 2021-02-20T13:19:46.433 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap2 2021-02-20T13:19:47.359 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Removing snap: 100% complete...done. 2021-02-20T13:19:47.443 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap2-clean 2021-02-20T13:19:47.443 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_snapshot cluster2 mirror test snap2-clean 2021-02-20T13:19:47.444 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:19:47.444 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:47.444 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:47.444 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap=snap2-clean 2021-02-20T13:19:47.445 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap2-clean 2021-02-20T13:19:48.388 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Removing snap: 100% complete...done. 2021-02-20T13:19:48.422 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap3 2021-02-20T13:19:48.423 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_snapshot cluster2 mirror test snap3 2021-02-20T13:19:48.423 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:19:48.423 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:48.424 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:48.424 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap=snap3 2021-02-20T13:19:48.424 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap3 2021-02-20T13:19:49.442 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Removing snap: 100% complete...done. 2021-02-20T13:19:49.477 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap3-clean 2021-02-20T13:19:49.477 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_snapshot cluster2 mirror test snap3-clean 2021-02-20T13:19:49.478 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:19:49.478 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:49.478 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:49.478 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap=snap3-clean 2021-02-20T13:19:49.479 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap3-clean 2021-02-20T13:19:50.384 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Removing snap: 100% complete...done. 2021-02-20T13:19:50.440 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap4 2021-02-20T13:19:50.440 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_snapshot cluster2 mirror test snap4 2021-02-20T13:19:50.440 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:19:50.440 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:50.441 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:50.441 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap=snap4 2021-02-20T13:19:50.441 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap4 2021-02-20T13:19:51.383 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Removing snap: 100% complete...done. 2021-02-20T13:19:51.430 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap4-clean 2021-02-20T13:19:51.430 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_snapshot cluster2 mirror test snap4-clean 2021-02-20T13:19:51.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:19:51.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:51.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:51.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap=snap4-clean 2021-02-20T13:19:51.432 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap4-clean 2021-02-20T13:19:52.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Removing snap: 100% complete...done. 2021-02-20T13:19:52.429 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap5 2021-02-20T13:19:52.429 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_snapshot cluster2 mirror test snap5 2021-02-20T13:19:52.429 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:19:52.429 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:52.430 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:52.430 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap=snap5 2021-02-20T13:19:52.430 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap5 2021-02-20T13:19:53.467 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Removing snap: 100% complete...done. 2021-02-20T13:19:53.625 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap5-clean 2021-02-20T13:19:53.626 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_snapshot cluster2 mirror test snap5-clean 2021-02-20T13:19:53.626 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:19:53.626 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:53.627 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:53.627 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap=snap5-clean 2021-02-20T13:19:53.628 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap5-clean 2021-02-20T13:19:54.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Removing snap: 100% complete...done. 2021-02-20T13:19:54.424 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap6 2021-02-20T13:19:54.424 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_snapshot cluster2 mirror test snap6 2021-02-20T13:19:54.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:19:54.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:54.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:54.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap=snap6 2021-02-20T13:19:54.426 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap6 2021-02-20T13:19:55.246 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Removing snap: 100% complete...done. 2021-02-20T13:19:55.302 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap6-clean 2021-02-20T13:19:55.302 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_snapshot cluster2 mirror test snap6-clean 2021-02-20T13:19:55.303 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:19:55.303 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:55.303 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:55.303 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap=snap6-clean 2021-02-20T13:19:55.304 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap6-clean 2021-02-20T13:19:56.406 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Removing snap: 100% complete...done. 2021-02-20T13:19:56.442 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap7 2021-02-20T13:19:56.443 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_snapshot cluster2 mirror test snap7 2021-02-20T13:19:56.443 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:19:56.443 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:56.443 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:56.444 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap=snap7 2021-02-20T13:19:56.444 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap7 2021-02-20T13:19:57.441 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Removing snap: 100% complete...done. 2021-02-20T13:19:57.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap7-clean 2021-02-20T13:19:57.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_snapshot cluster2 mirror test snap7-clean 2021-02-20T13:19:57.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:19:57.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:57.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:57.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap=snap7-clean 2021-02-20T13:19:57.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap7-clean 2021-02-20T13:19:58.402 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Removing snap: 100% complete...done. 2021-02-20T13:19:58.440 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap8 2021-02-20T13:19:58.441 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_snapshot cluster2 mirror test snap8 2021-02-20T13:19:58.441 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:19:58.441 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:58.441 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:58.442 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap=snap8 2021-02-20T13:19:58.442 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap8 2021-02-20T13:19:59.464 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Removing snap: 100% complete...done. 2021-02-20T13:19:59.502 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap8-clean 2021-02-20T13:19:59.502 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_snapshot cluster2 mirror test snap8-clean 2021-02-20T13:19:59.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:19:59.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:19:59.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:19:59.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap=snap8-clean 2021-02-20T13:19:59.504 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap8-clean 2021-02-20T13:20:00.235 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Removing snap: 100% complete...done. 2021-02-20T13:20:00.270 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap9 2021-02-20T13:20:00.270 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_snapshot cluster2 mirror test snap9 2021-02-20T13:20:00.270 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:20:00.271 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:00.271 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:20:00.271 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap=snap9 2021-02-20T13:20:00.271 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap9 2021-02-20T13:20:01.510 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Removing snap: 100% complete...done. 2021-02-20T13:20:01.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap9-clean 2021-02-20T13:20:01.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_snapshot cluster2 mirror test snap9-clean 2021-02-20T13:20:01.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:20:01.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:01.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:20:01.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap=snap9-clean 2021-02-20T13:20:01.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap9-clean 2021-02-20T13:20:02.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Removing snap: 100% complete...done. 2021-02-20T13:20:02.474 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap10 2021-02-20T13:20:02.474 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_snapshot cluster2 mirror test snap10 2021-02-20T13:20:02.474 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:20:02.475 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:02.475 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:20:02.475 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap=snap10 2021-02-20T13:20:02.475 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap10 2021-02-20T13:20:03.502 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Removing snap: 100% complete...done. 2021-02-20T13:20:03.572 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap10-clean 2021-02-20T13:20:03.573 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_snapshot cluster2 mirror test snap10-clean 2021-02-20T13:20:03.573 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:20:03.574 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:03.574 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:20:03.575 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local snap=snap10-clean 2021-02-20T13:20:03.575 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap10-clean 2021-02-20T13:20:04.461 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Removing snap: 100% complete...done. 2021-02-20T13:20:04.475 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_image_retry cluster2 mirror test 2021-02-20T13:20:04.476 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:20:04.476 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:04.476 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:20:04.476 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0 2021-02-20T13:20:04.477 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ remove_image cluster2 mirror test 2021-02-20T13:20:04.477 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:20:04.477 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:04.477 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:20:04.478 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 snap purge mirror/test 2021-02-20T13:20:04.543 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 rm mirror/test 2021-02-20T13:20:06.636 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2021-02-20T13:20:06.648 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:20:06.648 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ wait_for_image_present cluster1 mirror test deleted 2021-02-20T13:20:06.649 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:20:06.649 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:06.650 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:20:06.650 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local state=deleted 2021-02-20T13:20:06.661 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image_id= 2021-02-20T13:20:06.661 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local s 2021-02-20T13:20:06.661 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -n 2021-02-20T13:20:06.662 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_image_id cluster1 mirror test 2021-02-20T13:20:06.662 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:20:06.662 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:06.663 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:20:06.663 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-02-20T13:20:06.663 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-02-20T13:20:06.709 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image_id=109d36d23f3 2021-02-20T13:20:06.709 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 0.1 2021-02-20T13:20:06.811 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_image_present cluster1 mirror test deleted 109d36d23f3 2021-02-20T13:20:06.811 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:20:06.812 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:06.812 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:20:06.812 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local test_state=deleted 2021-02-20T13:20:06.812 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image_id=109d36d23f3 2021-02-20T13:20:06.812 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_state=deleted 2021-02-20T13:20:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_image_id 2021-02-20T13:20:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_image_id cluster1 mirror test 2021-02-20T13:20:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:20:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:20:06.814 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-02-20T13:20:06.814 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-02-20T13:20:06.885 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_image_id=109d36d23f3 2021-02-20T13:20:06.885 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -n 109d36d23f3 2021-02-20T13:20:06.885 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -z 109d36d23f3 -o 109d36d23f3 = 109d36d23f3 2021-02-20T13:20:06.886 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_state=present 2021-02-20T13:20:06.886 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test deleted = present 2021-02-20T13:20:06.886 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 1 2021-02-20T13:20:07.887 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_image_present cluster1 mirror test deleted 109d36d23f3 2021-02-20T13:20:07.887 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:20:07.887 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:07.887 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:20:07.888 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local test_state=deleted 2021-02-20T13:20:07.888 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image_id=109d36d23f3 2021-02-20T13:20:07.888 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_state=deleted 2021-02-20T13:20:07.888 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_image_id 2021-02-20T13:20:07.889 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_image_id cluster1 mirror test 2021-02-20T13:20:07.889 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:20:07.889 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:07.889 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:20:07.889 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-02-20T13:20:07.890 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-02-20T13:20:07.967 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_image_id=109d36d23f3 2021-02-20T13:20:07.967 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -n 109d36d23f3 2021-02-20T13:20:07.968 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -z 109d36d23f3 -o 109d36d23f3 = 109d36d23f3 2021-02-20T13:20:07.968 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_state=present 2021-02-20T13:20:07.969 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test deleted = present 2021-02-20T13:20:07.969 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 2 2021-02-20T13:20:09.970 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_image_present cluster1 mirror test deleted 109d36d23f3 2021-02-20T13:20:09.970 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:20:09.971 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:09.971 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:20:09.971 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local test_state=deleted 2021-02-20T13:20:09.972 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image_id=109d36d23f3 2021-02-20T13:20:09.972 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_state=deleted 2021-02-20T13:20:09.972 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_image_id 2021-02-20T13:20:09.973 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_image_id cluster1 mirror test 2021-02-20T13:20:09.973 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:20:09.973 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:09.974 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:20:09.974 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-02-20T13:20:09.975 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-02-20T13:20:10.042 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_image_id=109d36d23f3 2021-02-20T13:20:10.043 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -n 109d36d23f3 2021-02-20T13:20:10.043 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -z 109d36d23f3 -o 109d36d23f3 = 109d36d23f3 2021-02-20T13:20:10.043 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_state=present 2021-02-20T13:20:10.044 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test deleted = present 2021-02-20T13:20:10.044 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 4 2021-02-20T13:20:14.044 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_image_present cluster1 mirror test deleted 109d36d23f3 2021-02-20T13:20:14.045 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:20:14.045 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:14.045 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:20:14.045 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local test_state=deleted 2021-02-20T13:20:14.045 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image_id=109d36d23f3 2021-02-20T13:20:14.046 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_state=deleted 2021-02-20T13:20:14.046 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_image_id 2021-02-20T13:20:14.046 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_image_id cluster1 mirror test 2021-02-20T13:20:14.046 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:20:14.047 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:14.047 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:20:14.047 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-02-20T13:20:14.047 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-02-20T13:20:14.122 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_image_id=109d36d23f3 2021-02-20T13:20:14.122 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -n 109d36d23f3 2021-02-20T13:20:14.123 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -z 109d36d23f3 -o 109d36d23f3 = 109d36d23f3 2021-02-20T13:20:14.123 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_state=present 2021-02-20T13:20:14.123 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test deleted = present 2021-02-20T13:20:14.123 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 8 2021-02-20T13:20:22.124 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_image_present cluster1 mirror test deleted 109d36d23f3 2021-02-20T13:20:22.125 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:20:22.125 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:22.125 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:20:22.125 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local test_state=deleted 2021-02-20T13:20:22.126 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image_id=109d36d23f3 2021-02-20T13:20:22.126 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_state=deleted 2021-02-20T13:20:22.126 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_image_id 2021-02-20T13:20:22.126 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_image_id cluster1 mirror test 2021-02-20T13:20:22.127 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:20:22.127 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:22.127 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:20:22.127 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-02-20T13:20:22.127 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-02-20T13:20:22.194 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_image_id=109d36d23f3 2021-02-20T13:20:22.194 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -n 109d36d23f3 2021-02-20T13:20:22.194 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -z 109d36d23f3 -o 109d36d23f3 = 109d36d23f3 2021-02-20T13:20:22.195 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_state=present 2021-02-20T13:20:22.195 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test deleted = present 2021-02-20T13:20:22.195 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sleep 8 2021-02-20T13:20:30.196 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test_image_present cluster1 mirror test deleted 109d36d23f3 2021-02-20T13:20:30.197 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:20:30.197 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:30.197 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:20:30.197 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local test_state=deleted 2021-02-20T13:20:30.198 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image_id=109d36d23f3 2021-02-20T13:20:30.198 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_state=deleted 2021-02-20T13:20:30.198 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local current_image_id 2021-02-20T13:20:30.198 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ get_image_id cluster1 mirror test 2021-02-20T13:20:30.199 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster1 2021-02-20T13:20:30.199 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:30.199 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=test 2021-02-20T13:20:30.199 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-02-20T13:20:30.199 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-02-20T13:20:30.239 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:rbd: error opening image test: (2) No such file or directory 2021-02-20T13:20:30.243 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ current_image_id= 2021-02-20T13:20:30.244 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test -n 2021-02-20T13:20:30.244 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ test deleted = deleted 2021-02-20T13:20:30.245 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:20:30.245 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ testlog TEST: create many images 2021-02-20T13:20:30.245 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ tee -a /tmp/tmp.CsCtSCD9Zs/rbd-mirror.test.log 2021-02-20T13:20:30.246 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ date +%F %T 2021-02-20T13:20:30.248 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ echo 2021-02-20 13:20:30 TEST: create many images 2021-02-20T13:20:30.248 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:2021-02-20 13:20:30 TEST: create many images 2021-02-20T13:20:30.249 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ snap_name=snap 2021-02-20T13:20:30.249 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ seq 1 50 2021-02-20T13:20:30.249 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_1 2021-02-20T13:20:30.249 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_1 snapshot 128M 2021-02-20T13:20:30.250 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:20:30.250 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:30.250 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:30.250 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:30.251 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_1 2021-02-20T13:20:30.251 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:30.251 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:20:30.251 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:20:30.251 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:30.252 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_1 128M 2021-02-20T13:20:30.252 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:20:30.252 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:30.252 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:30.252 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:30.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_1 2021-02-20T13:20:30.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:30.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:20:30.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:20:30.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:20:30.254 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:30.254 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_1 2021-02-20T13:20:30.319 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:20:30.319 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_1 snapshot 2021-02-20T13:20:30.320 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:20:30.320 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:30.320 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_1 2021-02-20T13:20:30.320 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:20:30.320 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_1 snapshot 2021-02-20T13:20:31.379 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:20:31.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:20:31.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_1 2021-02-20T13:20:31.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:20:31.397 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:31.397 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_1 2021-02-20T13:20:31.398 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:20:31.399 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=13 2021-02-20T13:20:31.399 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:20:31.399 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_1 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:20:31.643 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 10620928 bytes 2021-02-20T13:20:31.902 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 44964864 bytes 2021-02-20T13:20:32.168 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 81586176 bytes 2021-02-20T13:20:32.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 111903744 bytes 2021-02-20T13:20:33.089 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 184364032 bytes 2021-02-20T13:20:33.246 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 187407360 bytes 2021-02-20T13:20:33.719 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 238537728 bytes 2021-02-20T13:20:33.914 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 248431616 bytes 2021-02-20T13:20:34.768 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 383498240 bytes 2021-02-20T13:20:35.222 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 441836544 bytes 2021-02-20T13:20:35.792 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 536428544 bytes 2021-02-20T13:20:36.304 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 570042368 bytes 2021-02-20T13:20:36.438 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 570681344 bytes 2021-02-20T13:20:37.006 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 644829184 bytes 2021-02-20T13:20:37.398 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 698040320 bytes 2021-02-20T13:20:38.035 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 782481408 bytes 2021-02-20T13:20:38.842 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 897923072 bytes 2021-02-20T13:20:39.230 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:20:39.231 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:20:39.232 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T13:20:39.233 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T13:20:39.231+0000 7fecf9776700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 28026) UID: 1000 2021-02-20T13:20:39.343 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 960100352 bytes 2021-02-20T13:20:39.777 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 1014766592 bytes 2021-02-20T13:20:39.836 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:20:40.232 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:20:40.233 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:20:40.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:20:40.234 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:20:40.234 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:20:40.234 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:20:40.236 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:20:40.237 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T13:20:40.271 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 1075225600 bytes 2021-02-20T13:20:40.649 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 1111649280 bytes 2021-02-20T13:20:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 1166420992 bytes 2021-02-20T13:20:41.443 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 1190626304 bytes 2021-02-20T13:20:41.760 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 1226383360 bytes 2021-02-20T13:20:42.146 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 1286591488 bytes 2021-02-20T13:20:42.474 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 1321953280 bytes 2021-02-20T13:20:42.992 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 1385767936 bytes 2021-02-20T13:20:43.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2800 IOs, 1418354688 bytes 2021-02-20T13:20:43.660 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2900 IOs, 1440705536 bytes 2021-02-20T13:20:43.980 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3000 IOs, 1475718144 bytes 2021-02-20T13:20:44.186 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3100 IOs, 1487580160 bytes 2021-02-20T13:20:44.416 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:20:44.416 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:20:44.417 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:20:44.417 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:20:44.417 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_2 2021-02-20T13:20:44.417 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_2 snapshot 128M 2021-02-20T13:20:44.417 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:20:44.418 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:44.418 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:44.418 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:44.418 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_2 2021-02-20T13:20:44.419 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:44.419 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:20:44.419 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:20:44.419 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:44.419 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_2 128M 2021-02-20T13:20:44.420 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:20:44.420 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:44.420 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:44.420 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:44.421 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_2 2021-02-20T13:20:44.421 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:44.421 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:20:44.421 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:20:44.422 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:20:44.422 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:44.422 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_2 2021-02-20T13:20:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:20:44.630 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_2 snapshot 2021-02-20T13:20:44.630 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:20:44.630 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:44.630 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_2 2021-02-20T13:20:44.631 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:20:44.631 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_2 snapshot 2021-02-20T13:20:45.405 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:20:45.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:20:45.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_2 2021-02-20T13:20:45.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:20:45.426 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:45.426 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_2 2021-02-20T13:20:45.426 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:20:45.484 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=14 2021-02-20T13:20:45.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:20:45.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_2 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:20:46.350 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 150145024 bytes 2021-02-20T13:20:46.727 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 181157888 bytes 2021-02-20T13:20:46.913 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 197784576 bytes 2021-02-20T13:20:47.388 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 237516800 bytes 2021-02-20T13:20:47.627 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 254831616 bytes 2021-02-20T13:20:48.064 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 312107008 bytes 2021-02-20T13:20:48.248 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 312701952 bytes 2021-02-20T13:20:48.471 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 330499072 bytes 2021-02-20T13:20:48.812 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 358748160 bytes 2021-02-20T13:20:49.102 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 379439104 bytes 2021-02-20T13:20:49.612 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 444178432 bytes 2021-02-20T13:20:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 455875584 bytes 2021-02-20T13:20:50.125 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 479027200 bytes 2021-02-20T13:20:50.850 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 584887296 bytes 2021-02-20T13:20:51.226 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 617190400 bytes 2021-02-20T13:20:51.695 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 669846528 bytes 2021-02-20T13:20:52.116 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 707470336 bytes 2021-02-20T13:20:52.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 754912256 bytes 2021-02-20T13:20:53.121 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 824395776 bytes 2021-02-20T13:20:53.274 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 825022464 bytes 2021-02-20T13:20:53.418 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 825656320 bytes 2021-02-20T13:20:53.795 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 869773312 bytes 2021-02-20T13:20:54.286 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 933060608 bytes 2021-02-20T13:20:54.609 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 966123520 bytes 2021-02-20T13:20:54.942 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 995763200 bytes 2021-02-20T13:20:55.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 1068008448 bytes 2021-02-20T13:20:55.989 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 1116918784 bytes 2021-02-20T13:20:56.279 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2800 IOs, 1142790144 bytes 2021-02-20T13:20:56.687 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2900 IOs, 1195196416 bytes 2021-02-20T13:20:57.273 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3000 IOs, 1278907392 bytes 2021-02-20T13:20:57.606 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3100 IOs, 1314153472 bytes 2021-02-20T13:20:58.071 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3200 IOs, 1385376768 bytes 2021-02-20T13:20:58.358 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3300 IOs, 1406824448 bytes 2021-02-20T13:20:58.523 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3400 IOs, 1407428608 bytes 2021-02-20T13:20:58.704 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3500 IOs, 1412667392 bytes 2021-02-20T13:20:59.095 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3600 IOs, 1481289728 bytes 2021-02-20T13:20:59.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3700 IOs, 1481913344 bytes 2021-02-20T13:20:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:20:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:20:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:20:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:20:59.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_3 2021-02-20T13:20:59.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_3 snapshot 128M 2021-02-20T13:20:59.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:20:59.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:59.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:59.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:59.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_3 2021-02-20T13:20:59.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:59.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:20:59.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:20:59.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:59.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_3 128M 2021-02-20T13:20:59.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:20:59.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:59.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:59.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:59.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_3 2021-02-20T13:20:59.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:59.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:20:59.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:20:59.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:20:59.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:20:59.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_3 2021-02-20T13:20:59.705 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:20:59.705 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_3 snapshot 2021-02-20T13:20:59.705 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:20:59.706 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:20:59.706 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_3 2021-02-20T13:20:59.706 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:20:59.706 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_3 snapshot 2021-02-20T13:21:00.384 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:21:00.400 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:21:00.401 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_3 2021-02-20T13:21:00.401 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:21:00.401 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:21:00.402 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_3 2021-02-20T13:21:00.402 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:21:00.403 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=12 2021-02-20T13:21:00.403 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:21:00.403 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_3 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:21:01.050 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 82488320 bytes 2021-02-20T13:21:01.650 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 144582656 bytes 2021-02-20T13:21:01.885 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 163952640 bytes 2021-02-20T13:21:02.576 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 261505024 bytes 2021-02-20T13:21:02.937 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 293198848 bytes 2021-02-20T13:21:03.094 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 293806080 bytes 2021-02-20T13:21:03.706 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 386682880 bytes 2021-02-20T13:21:04.114 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 423604224 bytes 2021-02-20T13:21:04.376 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 439393280 bytes 2021-02-20T13:21:04.774 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 479880192 bytes 2021-02-20T13:21:05.051 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 501892096 bytes 2021-02-20T13:21:05.202 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 502505472 bytes 2021-02-20T13:21:05.368 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 503124992 bytes 2021-02-20T13:21:05.975 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 574161920 bytes 2021-02-20T13:21:06.131 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 574743552 bytes 2021-02-20T13:21:06.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 602468352 bytes 2021-02-20T13:21:06.875 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 634954752 bytes 2021-02-20T13:21:07.262 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 666602496 bytes 2021-02-20T13:21:08.069 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 779855872 bytes 2021-02-20T13:21:08.269 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 780469248 bytes 2021-02-20T13:21:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 937710592 bytes 2021-02-20T13:21:09.638 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 972150784 bytes 2021-02-20T13:21:10.094 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 1018382336 bytes 2021-02-20T13:21:10.270 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 1025310720 bytes 2021-02-20T13:21:11.045 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 1097541632 bytes 2021-02-20T13:21:11.335 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 1118361600 bytes 2021-02-20T13:21:11.993 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 1189929984 bytes 2021-02-20T13:21:12.253 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2800 IOs, 1204184064 bytes 2021-02-20T13:21:12.408 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:21:12.408 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:21:12.408 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:21:12.409 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:21:12.409 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_4 2021-02-20T13:21:12.409 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_4 snapshot 128M 2021-02-20T13:21:12.410 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:21:12.410 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:12.410 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:21:12.410 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:12.411 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_4 2021-02-20T13:21:12.411 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:12.411 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:21:12.411 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:21:12.412 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:12.412 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_4 128M 2021-02-20T13:21:12.412 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:21:12.412 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:12.414 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:21:12.414 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:12.414 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_4 2021-02-20T13:21:12.415 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:12.415 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:21:12.415 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:21:12.416 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:21:12.416 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:12.416 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_4 2021-02-20T13:21:12.704 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:21:12.704 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_4 snapshot 2021-02-20T13:21:12.704 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:21:12.705 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:21:12.705 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_4 2021-02-20T13:21:12.705 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:21:12.705 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_4 snapshot 2021-02-20T13:21:13.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:21:13.455 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:21:13.455 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_4 2021-02-20T13:21:13.455 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:21:13.455 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:21:13.456 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_4 2021-02-20T13:21:13.456 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:21:13.457 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=5 2021-02-20T13:21:13.457 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:21:13.457 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_4 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:21:13.904 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 52600832 bytes 2021-02-20T13:21:14.381 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 115037184 bytes 2021-02-20T13:21:14.757 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 157198336 bytes 2021-02-20T13:21:15.085 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 211040256 bytes 2021-02-20T13:21:15.796 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 287750144 bytes 2021-02-20T13:21:16.189 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 341665792 bytes 2021-02-20T13:21:16.434 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 355038208 bytes 2021-02-20T13:21:16.635 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 400695296 bytes 2021-02-20T13:21:17.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 460138496 bytes 2021-02-20T13:21:17.785 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 500878336 bytes 2021-02-20T13:21:18.367 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 572104704 bytes 2021-02-20T13:21:18.462 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:21:18.463 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:21:18.463 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:21:18.463 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:21:18.463 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_5 2021-02-20T13:21:18.464 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_5 snapshot 128M 2021-02-20T13:21:18.464 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:21:18.464 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:18.465 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:21:18.465 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:18.465 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_5 2021-02-20T13:21:18.466 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:18.466 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:21:18.466 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:21:18.467 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:18.467 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_5 128M 2021-02-20T13:21:18.467 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:21:18.467 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:18.468 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:21:18.468 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:18.468 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_5 2021-02-20T13:21:18.468 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:18.469 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:21:18.469 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:21:18.469 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:21:18.469 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:18.470 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_5 2021-02-20T13:21:18.579 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:21:18.580 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_5 snapshot 2021-02-20T13:21:18.580 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:21:18.580 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:21:18.580 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_5 2021-02-20T13:21:18.581 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:21:18.581 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_5 snapshot 2021-02-20T13:21:19.402 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:21:19.419 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:21:19.419 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_5 2021-02-20T13:21:19.420 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:21:19.420 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:21:19.420 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_5 2021-02-20T13:21:19.420 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:21:19.423 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=14 2021-02-20T13:21:19.423 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:21:19.423 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_5 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:21:19.694 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 10335232 bytes 2021-02-20T13:21:20.018 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 45825024 bytes 2021-02-20T13:21:20.463 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 100550656 bytes 2021-02-20T13:21:21.008 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 173480960 bytes 2021-02-20T13:21:21.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 211290112 bytes 2021-02-20T13:21:21.898 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 276782080 bytes 2021-02-20T13:21:22.290 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 291901440 bytes 2021-02-20T13:21:22.553 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 307487744 bytes 2021-02-20T13:21:22.848 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 332849152 bytes 2021-02-20T13:21:23.402 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 407874560 bytes 2021-02-20T13:21:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 488196096 bytes 2021-02-20T13:21:24.339 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 514641920 bytes 2021-02-20T13:21:24.678 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 541578240 bytes 2021-02-20T13:21:24.907 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 554621952 bytes 2021-02-20T13:21:25.244 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 595461120 bytes 2021-02-20T13:21:25.261 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7537698150109452) > weight (0.25) 2021-02-20T13:21:25.261 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3725093972891331) > weight (0.25) 2021-02-20T13:21:25.262 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.802980325082835) > weight (0.25) 2021-02-20T13:21:25.262 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8579070241059045) > weight (0.25) 2021-02-20T13:21:25.262 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T13:21:25.447 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 610522112 bytes 2021-02-20T13:21:25.777 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 652869632 bytes 2021-02-20T13:21:26.269 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 701446144 bytes 2021-02-20T13:21:26.873 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 797353984 bytes 2021-02-20T13:21:27.898 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 875623424 bytes 2021-02-20T13:21:28.164 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 891848704 bytes 2021-02-20T13:21:28.630 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 962000896 bytes 2021-02-20T13:21:28.949 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 996525056 bytes 2021-02-20T13:21:29.465 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 1063820288 bytes 2021-02-20T13:21:30.083 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 1142292480 bytes 2021-02-20T13:21:30.290 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 1149037568 bytes 2021-02-20T13:21:30.878 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 1217904640 bytes 2021-02-20T13:21:31.321 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2800 IOs, 1264635904 bytes 2021-02-20T13:21:31.704 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2900 IOs, 1305184256 bytes 2021-02-20T13:21:32.188 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3000 IOs, 1364036608 bytes 2021-02-20T13:21:32.516 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3100 IOs, 1392613376 bytes 2021-02-20T13:21:32.747 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3200 IOs, 1402240000 bytes 2021-02-20T13:21:33.412 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3300 IOs, 1494630400 bytes 2021-02-20T13:21:33.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:21:33.426 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:21:33.426 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:21:33.426 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:21:33.427 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_6 2021-02-20T13:21:33.427 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_6 snapshot 128M 2021-02-20T13:21:33.427 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:21:33.427 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:33.427 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:21:33.428 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:33.428 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_6 2021-02-20T13:21:33.428 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:33.428 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:21:33.429 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:21:33.429 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:33.429 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_6 128M 2021-02-20T13:21:33.429 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:21:33.429 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:33.430 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:21:33.430 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:33.430 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_6 2021-02-20T13:21:33.430 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:33.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:21:33.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:21:33.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:21:33.432 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:33.432 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_6 2021-02-20T13:21:33.584 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:21:33.585 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_6 snapshot 2021-02-20T13:21:33.585 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:21:33.585 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:21:33.585 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_6 2021-02-20T13:21:33.586 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:21:33.586 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_6 snapshot 2021-02-20T13:21:34.406 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:21:34.428 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:21:34.428 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_6 2021-02-20T13:21:34.428 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:21:34.429 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:21:34.429 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_6 2021-02-20T13:21:34.429 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:21:34.430 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=5 2021-02-20T13:21:34.430 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:21:34.430 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_6 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:21:34.667 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 8045568 bytes 2021-02-20T13:21:35.076 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 55043072 bytes 2021-02-20T13:21:35.480 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 103973888 bytes 2021-02-20T13:21:35.761 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 132987904 bytes 2021-02-20T13:21:36.044 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 150306816 bytes 2021-02-20T13:21:36.524 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 202692608 bytes 2021-02-20T13:21:36.798 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 227281920 bytes 2021-02-20T13:21:37.084 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 249897984 bytes 2021-02-20T13:21:37.632 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 288518144 bytes 2021-02-20T13:21:37.943 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 318172160 bytes 2021-02-20T13:21:38.110 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 318782464 bytes 2021-02-20T13:21:38.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 366528512 bytes 2021-02-20T13:21:38.893 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 403295232 bytes 2021-02-20T13:21:39.244 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 462294016 bytes 2021-02-20T13:21:39.435 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:21:39.435 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:21:39.436 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:21:39.436 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:21:39.436 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_7 2021-02-20T13:21:39.437 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_7 snapshot 128M 2021-02-20T13:21:39.437 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:21:39.438 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:39.438 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:21:39.438 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:39.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_7 2021-02-20T13:21:39.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:39.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:21:39.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:21:39.440 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:39.440 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_7 128M 2021-02-20T13:21:39.440 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:21:39.440 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:39.441 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:21:39.441 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:39.442 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_7 2021-02-20T13:21:39.443 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:39.443 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:21:39.443 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:21:39.444 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:21:39.444 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:39.444 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_7 2021-02-20T13:21:39.556 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:21:39.556 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_7 snapshot 2021-02-20T13:21:39.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:21:39.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:21:39.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_7 2021-02-20T13:21:39.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:21:39.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_7 snapshot 2021-02-20T13:21:40.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:21:40.449 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:21:40.449 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_7 2021-02-20T13:21:40.450 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:21:40.450 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:21:40.450 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_7 2021-02-20T13:21:40.450 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:21:40.451 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=5 2021-02-20T13:21:40.451 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:21:40.452 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_7 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:21:41.081 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 68617216 bytes 2021-02-20T13:21:41.341 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 116361216 bytes 2021-02-20T13:21:41.904 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 172869632 bytes 2021-02-20T13:21:42.286 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 212283392 bytes 2021-02-20T13:21:42.881 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 310988800 bytes 2021-02-20T13:21:43.452 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 361413632 bytes 2021-02-20T13:21:43.798 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 368695296 bytes 2021-02-20T13:21:44.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 443017216 bytes 2021-02-20T13:21:44.751 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 503917568 bytes 2021-02-20T13:21:45.227 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 550623232 bytes 2021-02-20T13:21:45.408 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 554759168 bytes 2021-02-20T13:21:45.460 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:21:45.461 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:21:45.461 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:21:45.461 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:21:45.462 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_8 2021-02-20T13:21:45.462 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_8 snapshot 128M 2021-02-20T13:21:45.463 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:21:45.463 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:45.463 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:21:45.464 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:45.465 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_8 2021-02-20T13:21:45.465 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:21:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:21:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:45.469 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_8 128M 2021-02-20T13:21:45.469 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:21:45.469 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:45.469 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:21:45.470 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:45.470 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_8 2021-02-20T13:21:45.470 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:45.471 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:21:45.471 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:21:45.471 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:21:45.472 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:21:45.472 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_8 2021-02-20T13:21:45.899 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:21:45.899 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_8 snapshot 2021-02-20T13:21:45.899 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:21:45.899 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:21:45.900 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_8 2021-02-20T13:21:45.900 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:21:45.900 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_8 snapshot 2021-02-20T13:21:46.297 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:21:46.317 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:21:46.317 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_8 2021-02-20T13:21:46.317 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:21:46.318 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:21:46.318 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_8 2021-02-20T13:21:46.318 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:21:46.319 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=14 2021-02-20T13:21:46.320 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:21:46.320 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_8 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:21:46.777 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 58380288 bytes 2021-02-20T13:21:47.061 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 78871552 bytes 2021-02-20T13:21:47.790 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 149950464 bytes 2021-02-20T13:21:48.293 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 202893312 bytes 2021-02-20T13:21:48.891 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 286861312 bytes 2021-02-20T13:21:49.421 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 346034176 bytes 2021-02-20T13:21:49.800 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 400864256 bytes 2021-02-20T13:21:50.374 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 447176704 bytes 2021-02-20T13:21:50.944 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 516226048 bytes 2021-02-20T13:21:51.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 581135360 bytes 2021-02-20T13:21:52.018 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 662896640 bytes 2021-02-20T13:21:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 732288000 bytes 2021-02-20T13:21:53.328 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 802515968 bytes 2021-02-20T13:21:53.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 805669888 bytes 2021-02-20T13:21:54.075 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 880145408 bytes 2021-02-20T13:21:54.549 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 932501504 bytes 2021-02-20T13:21:55.032 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 995762176 bytes 2021-02-20T13:21:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 1045155840 bytes 2021-02-20T13:21:56.210 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 1121075200 bytes 2021-02-20T13:21:56.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 1154555904 bytes 2021-02-20T13:21:56.756 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 1156001792 bytes 2021-02-20T13:21:57.232 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 1210492928 bytes 2021-02-20T13:21:57.853 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 1298194432 bytes 2021-02-20T13:21:58.483 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 1368945664 bytes 2021-02-20T13:21:58.894 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 1421349888 bytes 2021-02-20T13:21:59.276 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 1456849920 bytes 2021-02-20T13:21:59.810 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 1549316096 bytes 2021-02-20T13:22:00.323 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:22:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:22:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:22:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:22:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_9 2021-02-20T13:22:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_9 snapshot 128M 2021-02-20T13:22:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:22:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:22:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_9 2021-02-20T13:22:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:22:00.327 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:22:00.327 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:00.327 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_9 128M 2021-02-20T13:22:00.327 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:22:00.327 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:00.328 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:22:00.328 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:00.328 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_9 2021-02-20T13:22:00.328 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:00.328 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:22:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:22:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:22:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_9 2021-02-20T13:22:00.397 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:22:00.398 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_9 snapshot 2021-02-20T13:22:00.398 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:22:00.398 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:22:00.398 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_9 2021-02-20T13:22:00.399 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:22:00.399 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_9 snapshot 2021-02-20T13:22:01.424 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:22:01.445 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:22:01.446 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_9 2021-02-20T13:22:01.446 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:22:01.446 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:22:01.447 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_9 2021-02-20T13:22:01.447 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:22:01.448 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=6 2021-02-20T13:22:01.449 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:22:01.449 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_9 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:22:01.714 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 16857088 bytes 2021-02-20T13:22:02.067 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 47632384 bytes 2021-02-20T13:22:02.359 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 78046208 bytes 2021-02-20T13:22:02.741 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 102285312 bytes 2021-02-20T13:22:03.240 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 144802816 bytes 2021-02-20T13:22:03.610 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 197881856 bytes 2021-02-20T13:22:03.970 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 212756480 bytes 2021-02-20T13:22:04.295 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 237658112 bytes 2021-02-20T13:22:04.464 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 241118208 bytes 2021-02-20T13:22:04.978 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 287849472 bytes 2021-02-20T13:22:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 332729344 bytes 2021-02-20T13:22:05.926 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 394723328 bytes 2021-02-20T13:22:06.443 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 450611200 bytes 2021-02-20T13:22:06.899 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 501952512 bytes 2021-02-20T13:22:07.265 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 535322624 bytes 2021-02-20T13:22:07.453 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:22:07.453 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:22:07.454 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:22:07.454 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:22:07.454 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_10 2021-02-20T13:22:07.455 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_10 snapshot 128M 2021-02-20T13:22:07.455 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:22:07.455 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:07.455 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:22:07.456 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:07.456 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_10 2021-02-20T13:22:07.456 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:07.456 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:22:07.457 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:22:07.457 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:07.457 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_10 128M 2021-02-20T13:22:07.457 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:22:07.458 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:07.458 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:22:07.458 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:07.458 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_10 2021-02-20T13:22:07.459 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:07.459 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:22:07.460 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:22:07.461 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:22:07.462 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:07.462 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_10 2021-02-20T13:22:07.522 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:22:07.523 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_10 snapshot 2021-02-20T13:22:07.523 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:22:07.524 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:22:07.524 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_10 2021-02-20T13:22:07.524 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:22:07.525 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_10 snapshot 2021-02-20T13:22:08.437 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:22:08.457 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:22:08.457 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_10 2021-02-20T13:22:08.458 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:22:08.458 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:22:08.458 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_10 2021-02-20T13:22:08.458 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:22:08.459 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=13 2021-02-20T13:22:08.459 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:22:08.460 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_10 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:22:09.108 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 89437184 bytes 2021-02-20T13:22:09.530 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 138674176 bytes 2021-02-20T13:22:09.822 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 160197632 bytes 2021-02-20T13:22:10.379 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 246204416 bytes 2021-02-20T13:22:11.063 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 335334400 bytes 2021-02-20T13:22:11.365 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 346462208 bytes 2021-02-20T13:22:11.698 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 378877952 bytes 2021-02-20T13:22:11.982 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 409183232 bytes 2021-02-20T13:22:12.386 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 443278336 bytes 2021-02-20T13:22:12.772 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 483295232 bytes 2021-02-20T13:22:13.020 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 497455104 bytes 2021-02-20T13:22:13.864 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 570184704 bytes 2021-02-20T13:22:14.064 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 572521472 bytes 2021-02-20T13:22:14.376 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 600744960 bytes 2021-02-20T13:22:14.841 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 647971840 bytes 2021-02-20T13:22:15.045 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 654751744 bytes 2021-02-20T13:22:15.498 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 704231424 bytes 2021-02-20T13:22:15.858 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 742738944 bytes 2021-02-20T13:22:16.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 820401152 bytes 2021-02-20T13:22:16.797 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 845587456 bytes 2021-02-20T13:22:17.622 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 981023744 bytes 2021-02-20T13:22:18.127 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 1019661312 bytes 2021-02-20T13:22:18.695 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 1060632576 bytes 2021-02-20T13:22:19.151 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 1143753728 bytes 2021-02-20T13:22:19.666 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 1172783104 bytes 2021-02-20T13:22:20.106 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 1229483008 bytes 2021-02-20T13:22:20.467 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 1251393536 bytes 2021-02-20T13:22:20.949 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2800 IOs, 1304957952 bytes 2021-02-20T13:22:21.328 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2900 IOs, 1343571968 bytes 2021-02-20T13:22:21.464 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:22:21.464 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:22:21.464 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:22:21.464 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:22:21.465 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_11 2021-02-20T13:22:21.465 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_11 snapshot 128M 2021-02-20T13:22:21.465 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:22:21.465 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:21.465 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:22:21.466 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:21.466 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_11 2021-02-20T13:22:21.466 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:21.466 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:22:21.467 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:22:21.467 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:21.467 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_11 128M 2021-02-20T13:22:21.467 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:22:21.467 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:21.468 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:22:21.468 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:21.468 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_11 2021-02-20T13:22:21.468 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:21.469 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:22:21.469 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:22:21.469 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:22:21.470 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:21.470 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_11 2021-02-20T13:22:21.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:22:21.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_11 snapshot 2021-02-20T13:22:21.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:22:21.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:22:21.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_11 2021-02-20T13:22:21.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:22:21.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_11 snapshot 2021-02-20T13:22:22.422 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:22:22.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:22:22.440 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_11 2021-02-20T13:22:22.440 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:22:22.440 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:22:22.440 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_11 2021-02-20T13:22:22.440 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:22:22.441 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=12 2021-02-20T13:22:22.441 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:22:22.442 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_11 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:22:22.859 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 24584192 bytes 2021-02-20T13:22:23.185 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 53352448 bytes 2021-02-20T13:22:23.894 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 153140224 bytes 2021-02-20T13:22:24.230 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 170465280 bytes 2021-02-20T13:22:24.739 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 234512384 bytes 2021-02-20T13:22:25.222 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 288989184 bytes 2021-02-20T13:22:25.604 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 328278016 bytes 2021-02-20T13:22:25.986 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 368245760 bytes 2021-02-20T13:22:26.350 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 407104512 bytes 2021-02-20T13:22:26.673 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 451084288 bytes 2021-02-20T13:22:27.009 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 474206208 bytes 2021-02-20T13:22:27.551 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 555163648 bytes 2021-02-20T13:22:28.029 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 601157632 bytes 2021-02-20T13:22:28.755 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 695037952 bytes 2021-02-20T13:22:29.007 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 695665664 bytes 2021-02-20T13:22:29.568 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 765296640 bytes 2021-02-20T13:22:29.836 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 801502208 bytes 2021-02-20T13:22:30.164 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 820488192 bytes 2021-02-20T13:22:30.789 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 904142848 bytes 2021-02-20T13:22:31.036 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 921202688 bytes 2021-02-20T13:22:31.429 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 969169920 bytes 2021-02-20T13:22:31.809 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 994717696 bytes 2021-02-20T13:22:32.218 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 1037554688 bytes 2021-02-20T13:22:32.631 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 1079902208 bytes 2021-02-20T13:22:33.026 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 1085080576 bytes 2021-02-20T13:22:33.672 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 1165593600 bytes 2021-02-20T13:22:34.144 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 1225850880 bytes 2021-02-20T13:22:34.527 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:22:34.527 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:22:34.529 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:22:34.529 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:22:34.529 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_12 2021-02-20T13:22:34.530 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_12 snapshot 128M 2021-02-20T13:22:34.531 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:22:34.531 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:34.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:22:34.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:34.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_12 2021-02-20T13:22:34.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:34.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:22:34.534 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:22:34.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:34.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_12 128M 2021-02-20T13:22:34.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:22:34.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:34.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:22:34.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:34.551 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_12 2021-02-20T13:22:34.552 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:34.552 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:22:34.552 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:22:34.553 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:22:34.553 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:34.554 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_12 2021-02-20T13:22:34.632 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:22:34.633 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_12 snapshot 2021-02-20T13:22:34.633 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:22:34.634 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:22:34.634 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_12 2021-02-20T13:22:34.635 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:22:34.635 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_12 snapshot 2021-02-20T13:22:35.579 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:22:35.608 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:22:35.608 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_12 2021-02-20T13:22:35.609 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:22:35.609 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:22:35.609 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_12 2021-02-20T13:22:35.610 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:22:35.612 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=11 2021-02-20T13:22:35.612 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:22:35.613 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_12 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:22:35.877 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 7319552 bytes 2021-02-20T13:22:36.079 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 20858880 bytes 2021-02-20T13:22:36.641 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 65313792 bytes 2021-02-20T13:22:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 119345152 bytes 2021-02-20T13:22:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 172016640 bytes 2021-02-20T13:22:37.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 192135168 bytes 2021-02-20T13:22:38.023 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 248950784 bytes 2021-02-20T13:22:38.233 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 256759808 bytes 2021-02-20T13:22:38.237 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.42849315935326815) > weight (0.25) 2021-02-20T13:22:38.244 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5350443900984788) > weight (0.25) 2021-02-20T13:22:38.245 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.36621215278692776) > weight (0.25) 2021-02-20T13:22:38.245 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9002604780317489) > weight (0.25) 2021-02-20T13:22:38.245 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T13:22:38.603 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 271882240 bytes 2021-02-20T13:22:39.169 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 353869824 bytes 2021-02-20T13:22:39.377 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 354480128 bytes 2021-02-20T13:22:39.844 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 411446272 bytes 2021-02-20T13:22:40.540 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 512440320 bytes 2021-02-20T13:22:40.809 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 519612416 bytes 2021-02-20T13:22:41.092 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 542155776 bytes 2021-02-20T13:22:41.317 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 554238976 bytes 2021-02-20T13:22:41.821 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 617020416 bytes 2021-02-20T13:22:42.204 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 656814080 bytes 2021-02-20T13:22:42.428 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 666014720 bytes 2021-02-20T13:22:42.741 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 712027136 bytes 2021-02-20T13:22:43.216 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 778936320 bytes 2021-02-20T13:22:43.977 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 816164864 bytes 2021-02-20T13:22:44.267 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 856281088 bytes 2021-02-20T13:22:44.879 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 914916352 bytes 2021-02-20T13:22:45.243 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 946759680 bytes 2021-02-20T13:22:45.663 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 980274176 bytes 2021-02-20T13:22:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 1032979456 bytes 2021-02-20T13:22:46.600 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2800 IOs, 1079654400 bytes 2021-02-20T13:22:46.611 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:22:46.611 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:22:46.612 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:22:46.612 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:22:46.612 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_13 2021-02-20T13:22:46.612 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_13 snapshot 128M 2021-02-20T13:22:46.613 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:22:46.613 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:46.613 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:22:46.613 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:46.614 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_13 2021-02-20T13:22:46.614 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:46.614 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:22:46.614 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:22:46.614 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:46.615 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_13 128M 2021-02-20T13:22:46.615 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:22:46.615 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:46.615 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:22:46.615 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:46.616 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_13 2021-02-20T13:22:46.616 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:46.616 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:22:46.616 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:22:46.617 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:22:46.617 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:22:46.617 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_13 2021-02-20T13:22:46.801 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:22:46.802 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_13 snapshot 2021-02-20T13:22:46.802 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:22:46.802 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:22:46.802 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_13 2021-02-20T13:22:46.803 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:22:46.803 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_13 snapshot 2021-02-20T13:22:47.312 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:22:47.340 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:22:47.340 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_13 2021-02-20T13:22:47.341 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:22:47.341 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:22:47.341 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_13 2021-02-20T13:22:47.341 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:22:47.342 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=13 2021-02-20T13:22:47.342 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:22:47.342 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_13 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:22:47.996 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 108381184 bytes 2021-02-20T13:22:48.400 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 148516864 bytes 2021-02-20T13:22:48.984 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 197191680 bytes 2021-02-20T13:22:49.359 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 242515968 bytes 2021-02-20T13:22:49.800 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 277628928 bytes 2021-02-20T13:22:50.643 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 390452224 bytes 2021-02-20T13:22:51.103 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 428565504 bytes 2021-02-20T13:22:51.555 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 474070016 bytes 2021-02-20T13:22:51.932 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 507741184 bytes 2021-02-20T13:22:52.543 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 584295424 bytes 2021-02-20T13:22:52.920 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 616800256 bytes 2021-02-20T13:22:53.355 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 670184448 bytes 2021-02-20T13:22:53.697 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 690935808 bytes 2021-02-20T13:22:54.105 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 730541056 bytes 2021-02-20T13:22:54.578 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 789664768 bytes 2021-02-20T13:22:55.007 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 825697280 bytes 2021-02-20T13:22:55.400 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 869377024 bytes 2021-02-20T13:22:55.912 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 935065600 bytes 2021-02-20T13:22:56.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 972112896 bytes 2021-02-20T13:22:56.651 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 981474304 bytes 2021-02-20T13:22:57.471 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 1063817216 bytes 2021-02-20T13:22:58.095 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 1141549056 bytes 2021-02-20T13:22:58.602 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 1199115264 bytes 2021-02-20T13:22:59.098 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 1205922816 bytes 2021-02-20T13:22:59.529 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 1224125440 bytes 2021-02-20T13:22:59.829 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 1243250688 bytes 2021-02-20T13:23:00.153 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 1271684096 bytes 2021-02-20T13:23:00.346 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:23:00.346 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:23:00.346 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:23:00.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:23:00.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_14 2021-02-20T13:23:00.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_14 snapshot 128M 2021-02-20T13:23:00.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:23:00.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:00.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:23:00.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:00.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_14 2021-02-20T13:23:00.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:00.350 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:23:00.351 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:23:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_14 128M 2021-02-20T13:23:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:23:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:23:00.353 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:00.353 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_14 2021-02-20T13:23:00.353 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:00.353 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:23:00.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:23:00.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:23:00.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:00.355 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_14 2021-02-20T13:23:00.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:23:00.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_14 snapshot 2021-02-20T13:23:00.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:23:00.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:23:00.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_14 2021-02-20T13:23:00.509 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:23:00.509 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_14 snapshot 2021-02-20T13:23:01.464 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:23:01.482 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:23:01.483 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_14 2021-02-20T13:23:01.483 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:23:01.483 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:23:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_14 2021-02-20T13:23:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:23:01.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=12 2021-02-20T13:23:01.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:23:01.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_14 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:23:01.748 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 619520 bytes 2021-02-20T13:23:02.126 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 55794688 bytes 2021-02-20T13:23:02.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 98620416 bytes 2021-02-20T13:23:02.880 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 133484544 bytes 2021-02-20T13:23:03.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 187906048 bytes 2021-02-20T13:23:03.982 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 244897792 bytes 2021-02-20T13:23:04.291 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 270996480 bytes 2021-02-20T13:23:04.806 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 314632192 bytes 2021-02-20T13:23:05.343 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 370278400 bytes 2021-02-20T13:23:05.796 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 416129024 bytes 2021-02-20T13:23:06.296 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 490309632 bytes 2021-02-20T13:23:07.142 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 581735424 bytes 2021-02-20T13:23:07.601 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 613387264 bytes 2021-02-20T13:23:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 626539520 bytes 2021-02-20T13:23:08.245 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 668558336 bytes 2021-02-20T13:23:08.715 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 699734016 bytes 2021-02-20T13:23:09.034 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 717440000 bytes 2021-02-20T13:23:09.357 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 741435392 bytes 2021-02-20T13:23:09.682 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 769849344 bytes 2021-02-20T13:23:09.867 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 770454528 bytes 2021-02-20T13:23:10.087 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 778066944 bytes 2021-02-20T13:23:10.764 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 874360832 bytes 2021-02-20T13:23:11.088 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 889733120 bytes 2021-02-20T13:23:11.542 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 937241600 bytes 2021-02-20T13:23:12.111 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 1012160512 bytes 2021-02-20T13:23:12.563 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 1055703040 bytes 2021-02-20T13:23:13.247 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 1123065856 bytes 2021-02-20T13:23:13.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:23:13.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:23:13.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:23:13.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:23:13.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_15 2021-02-20T13:23:13.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_15 snapshot 128M 2021-02-20T13:23:13.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:23:13.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:13.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:23:13.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:13.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_15 2021-02-20T13:23:13.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:13.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:23:13.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:23:13.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:13.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_15 128M 2021-02-20T13:23:13.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:23:13.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:13.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:23:13.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:13.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_15 2021-02-20T13:23:13.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:13.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:23:13.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:23:13.496 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:23:13.496 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:13.496 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_15 2021-02-20T13:23:13.637 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:23:13.638 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_15 snapshot 2021-02-20T13:23:13.638 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:23:13.638 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:23:13.638 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_15 2021-02-20T13:23:13.639 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:23:13.639 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_15 snapshot 2021-02-20T13:23:14.482 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:23:14.506 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:23:14.506 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_15 2021-02-20T13:23:14.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:23:14.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:23:14.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_15 2021-02-20T13:23:14.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:23:14.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=8 2021-02-20T13:23:14.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:23:14.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_15 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:23:15.047 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 72227840 bytes 2021-02-20T13:23:15.641 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 149030912 bytes 2021-02-20T13:23:16.152 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 209833984 bytes 2021-02-20T13:23:16.263 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8728905469185742) > weight (0.25) 2021-02-20T13:23:16.263 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T13:23:16.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T13:23:16.265 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T13:23:16.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:23:16.323+0000 7fad12bb1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 19132) UID: 1000 2021-02-20T13:23:16.801 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 314278912 bytes 2021-02-20T13:23:16.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:23:17.101 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 329714688 bytes 2021-02-20T13:23:17.570 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 403453952 bytes 2021-02-20T13:23:17.826 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 404074496 bytes 2021-02-20T13:23:18.137 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 448048128 bytes 2021-02-20T13:23:18.266 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T13:23:18.267 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:23:18.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:23:18.267 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T13:23:18.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T13:23:18.268 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T13:23:18.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T13:23:18.435 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T13:23:18.568 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 484119552 bytes 2021-02-20T13:23:19.062 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 548566016 bytes 2021-02-20T13:23:19.281 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 550320128 bytes 2021-02-20T13:23:19.612 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 580489216 bytes 2021-02-20T13:23:20.345 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 695350272 bytes 2021-02-20T13:23:20.835 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 739927040 bytes 2021-02-20T13:23:21.169 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 771917824 bytes 2021-02-20T13:23:21.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 819539968 bytes 2021-02-20T13:23:22.228 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 932209664 bytes 2021-02-20T13:23:22.513 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:23:22.514 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:23:22.515 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:23:22.515 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:23:22.516 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_16 2021-02-20T13:23:22.516 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_16 snapshot 128M 2021-02-20T13:23:22.517 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:23:22.518 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:22.519 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:23:22.519 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:22.519 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_16 2021-02-20T13:23:22.520 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:22.520 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:23:22.520 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:23:22.521 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:22.521 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_16 128M 2021-02-20T13:23:22.521 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:23:22.522 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:22.522 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:23:22.522 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:22.523 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_16 2021-02-20T13:23:22.523 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:22.523 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:23:22.523 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:23:22.524 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:23:22.524 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:22.524 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_16 2021-02-20T13:23:22.762 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:23:22.762 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_16 snapshot 2021-02-20T13:23:22.763 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:23:22.763 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:23:22.763 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_16 2021-02-20T13:23:22.764 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:23:22.764 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_16 snapshot 2021-02-20T13:23:23.476 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:23:23.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:23:23.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_16 2021-02-20T13:23:23.496 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:23:23.496 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:23:23.496 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_16 2021-02-20T13:23:23.496 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:23:23.584 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=12 2021-02-20T13:23:23.585 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:23:23.585 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_16 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:23:24.300 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 53494784 bytes 2021-02-20T13:23:24.837 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 125524992 bytes 2021-02-20T13:23:25.365 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 201682944 bytes 2021-02-20T13:23:25.611 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 215506944 bytes 2021-02-20T13:23:25.975 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 270198784 bytes 2021-02-20T13:23:26.289 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 281385984 bytes 2021-02-20T13:23:26.523 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 281982976 bytes 2021-02-20T13:23:26.862 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 320821248 bytes 2021-02-20T13:23:27.367 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 377339904 bytes 2021-02-20T13:23:27.982 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 448591872 bytes 2021-02-20T13:23:28.344 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 497212416 bytes 2021-02-20T13:23:28.826 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 534903808 bytes 2021-02-20T13:23:29.405 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 593036288 bytes 2021-02-20T13:23:29.932 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 655901696 bytes 2021-02-20T13:23:30.421 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 720707584 bytes 2021-02-20T13:23:31.144 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 828312576 bytes 2021-02-20T13:23:31.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 873598976 bytes 2021-02-20T13:23:32.098 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 936442880 bytes 2021-02-20T13:23:32.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 1019450368 bytes 2021-02-20T13:23:32.985 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 1045595136 bytes 2021-02-20T13:23:33.285 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 1060585472 bytes 2021-02-20T13:23:33.836 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 1129553920 bytes 2021-02-20T13:23:34.298 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 1159811072 bytes 2021-02-20T13:23:34.539 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 1164419072 bytes 2021-02-20T13:23:34.878 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 1193364480 bytes 2021-02-20T13:23:35.251 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 1226039296 bytes 2021-02-20T13:23:35.610 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 1263765504 bytes 2021-02-20T13:23:35.673 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:23:35.674 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:23:35.675 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:23:35.675 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:23:35.675 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_17 2021-02-20T13:23:35.675 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_17 snapshot 128M 2021-02-20T13:23:35.676 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:23:35.676 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:35.676 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:23:35.676 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:35.676 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_17 2021-02-20T13:23:35.677 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:35.677 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:23:35.678 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:23:35.678 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:35.679 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_17 128M 2021-02-20T13:23:35.679 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:23:35.679 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:35.679 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:23:35.680 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:35.680 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_17 2021-02-20T13:23:35.680 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:35.680 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:23:35.680 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:23:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:23:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_17 2021-02-20T13:23:35.912 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:23:35.912 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_17 snapshot 2021-02-20T13:23:35.913 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:23:35.913 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:23:35.914 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_17 2021-02-20T13:23:35.914 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:23:35.915 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_17 snapshot 2021-02-20T13:23:36.454 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:23:36.478 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:23:36.479 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_17 2021-02-20T13:23:36.479 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:23:36.479 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:23:36.480 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_17 2021-02-20T13:23:36.480 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:23:36.481 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=13 2021-02-20T13:23:36.481 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:23:36.481 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_17 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:23:36.972 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 27538432 bytes 2021-02-20T13:23:37.418 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 88061952 bytes 2021-02-20T13:23:37.677 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 107591680 bytes 2021-02-20T13:23:38.239 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 194518016 bytes 2021-02-20T13:23:38.546 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 214847488 bytes 2021-02-20T13:23:38.925 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 254675968 bytes 2021-02-20T13:23:39.553 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 300311552 bytes 2021-02-20T13:23:40.063 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 362382336 bytes 2021-02-20T13:23:40.566 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 438994944 bytes 2021-02-20T13:23:41.042 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 479406080 bytes 2021-02-20T13:23:41.636 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 559799296 bytes 2021-02-20T13:23:42.087 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 616767488 bytes 2021-02-20T13:23:42.556 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 660767744 bytes 2021-02-20T13:23:43.094 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 712374272 bytes 2021-02-20T13:23:43.745 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 808531968 bytes 2021-02-20T13:23:44.134 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 827600896 bytes 2021-02-20T13:23:44.761 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 867772416 bytes 2021-02-20T13:23:45.187 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 916544512 bytes 2021-02-20T13:23:45.631 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 981760000 bytes 2021-02-20T13:23:46.018 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 1008786432 bytes 2021-02-20T13:23:46.288 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 1027232768 bytes 2021-02-20T13:23:46.998 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 1153188864 bytes 2021-02-20T13:23:47.404 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 1174797312 bytes 2021-02-20T13:23:47.824 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 1217616896 bytes 2021-02-20T13:23:48.227 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 1259885568 bytes 2021-02-20T13:23:48.605 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 1303649280 bytes 2021-02-20T13:23:48.952 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 1337619456 bytes 2021-02-20T13:23:49.338 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2800 IOs, 1386960896 bytes 2021-02-20T13:23:49.588 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:23:49.588 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:23:49.588 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:23:49.589 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:23:49.589 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_18 2021-02-20T13:23:49.589 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_18 snapshot 128M 2021-02-20T13:23:49.589 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:23:49.589 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:49.590 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:23:49.590 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:49.590 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_18 2021-02-20T13:23:49.590 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:49.591 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:23:49.591 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:23:49.591 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:49.591 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_18 128M 2021-02-20T13:23:49.591 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:23:49.592 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:49.592 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:23:49.593 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:49.593 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_18 2021-02-20T13:23:49.593 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:49.593 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:23:49.594 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:23:49.594 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:23:49.594 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:23:49.594 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_18 2021-02-20T13:23:49.685 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:23:49.685 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_18 snapshot 2021-02-20T13:23:49.685 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:23:49.686 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:23:49.686 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_18 2021-02-20T13:23:49.686 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:23:49.686 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_18 snapshot 2021-02-20T13:23:50.466 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:23:50.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:23:50.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_18 2021-02-20T13:23:50.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:23:50.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:23:50.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_18 2021-02-20T13:23:50.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:23:50.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=10 2021-02-20T13:23:50.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:23:50.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_18 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:23:50.827 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 25456640 bytes 2021-02-20T13:23:51.127 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 51152896 bytes 2021-02-20T13:23:51.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 99799040 bytes 2021-02-20T13:23:51.868 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 141978624 bytes 2021-02-20T13:23:52.197 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 191780864 bytes 2021-02-20T13:23:52.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 227520512 bytes 2021-02-20T13:23:53.196 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 279144448 bytes 2021-02-20T13:23:53.665 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 334734336 bytes 2021-02-20T13:23:54.079 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 376390656 bytes 2021-02-20T13:23:54.309 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 406671360 bytes 2021-02-20T13:23:54.912 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 466316288 bytes 2021-02-20T13:23:55.250 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 490233856 bytes 2021-02-20T13:23:55.458 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 490869760 bytes 2021-02-20T13:23:55.665 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 493551616 bytes 2021-02-20T13:23:55.923 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 525513728 bytes 2021-02-20T13:23:56.474 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 572989440 bytes 2021-02-20T13:23:56.905 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 629153792 bytes 2021-02-20T13:23:57.458 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 689513472 bytes 2021-02-20T13:23:57.791 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 724015104 bytes 2021-02-20T13:23:58.226 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 772543488 bytes 2021-02-20T13:23:58.753 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 821524480 bytes 2021-02-20T13:23:59.068 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 854822912 bytes 2021-02-20T13:23:59.733 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 928721920 bytes 2021-02-20T13:24:00.167 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 962592768 bytes 2021-02-20T13:24:00.247 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8787335530380217) > weight (0.25) 2021-02-20T13:24:00.247 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:24:00.248 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:24:00.248 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T13:24:00.248 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T13:24:00.248 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:24:00.249 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T13:24:00.246+0000 7f787f61a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 24087) UID: 1000 2021-02-20T13:24:00.443 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 983488512 bytes 2021-02-20T13:24:00.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:24:00.496 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:24:00.496 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:24:00.496 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:24:00.496 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_19 2021-02-20T13:24:00.496 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_19 snapshot 128M 2021-02-20T13:24:00.497 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:00.497 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:00.497 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:00.497 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:00.497 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_19 2021-02-20T13:24:00.498 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:00.498 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:24:00.498 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:24:00.498 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:00.499 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_19 128M 2021-02-20T13:24:00.499 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:00.499 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:00.499 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:00.499 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:00.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_19 2021-02-20T13:24:00.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:00.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:24:00.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:24:00.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:24:00.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:00.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_19 2021-02-20T13:24:00.559 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:24:00.559 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_19 snapshot 2021-02-20T13:24:00.559 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:00.559 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:00.560 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_19 2021-02-20T13:24:00.560 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:24:00.560 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_19 snapshot 2021-02-20T13:24:00.649 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:24:01.467 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:24:01.756 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:24:01.756 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_19 2021-02-20T13:24:01.757 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:01.757 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:01.758 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_19 2021-02-20T13:24:01.758 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:24:01.758 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=10 2021-02-20T13:24:01.759 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:24:01.759 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_19 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:24:02.035 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 61255680 bytes 2021-02-20T13:24:02.323 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 86879232 bytes 2021-02-20T13:24:02.680 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 122863616 bytes 2021-02-20T13:24:02.873 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 123866112 bytes 2021-02-20T13:24:03.314 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 175453184 bytes 2021-02-20T13:24:03.709 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 226843648 bytes 2021-02-20T13:24:04.129 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 299689984 bytes 2021-02-20T13:24:04.436 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 319918080 bytes 2021-02-20T13:24:05.089 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 335846400 bytes 2021-02-20T13:24:05.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 359847936 bytes 2021-02-20T13:24:05.725 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 383630336 bytes 2021-02-20T13:24:06.031 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 430314496 bytes 2021-02-20T13:24:06.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:24:06.351 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T13:24:06.351 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:24:06.352 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:24:06.365 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:24:06.365 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T13:24:06.450 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 447899648 bytes 2021-02-20T13:24:06.892 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 489098240 bytes 2021-02-20T13:24:07.373 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 537958400 bytes 2021-02-20T13:24:07.650 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 558292992 bytes 2021-02-20T13:24:07.896 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 569181184 bytes 2021-02-20T13:24:08.515 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 672164864 bytes 2021-02-20T13:24:08.986 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 718364672 bytes 2021-02-20T13:24:09.340 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 752937984 bytes 2021-02-20T13:24:09.574 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 763547648 bytes 2021-02-20T13:24:10.044 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 812310528 bytes 2021-02-20T13:24:10.239 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 812926976 bytes 2021-02-20T13:24:10.736 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 867225600 bytes 2021-02-20T13:24:11.010 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 884321280 bytes 2021-02-20T13:24:11.319 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 909936640 bytes 2021-02-20T13:24:11.502 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:24:11.502 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:24:11.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:24:11.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:24:11.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_20 2021-02-20T13:24:11.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_20 snapshot 128M 2021-02-20T13:24:11.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:11.504 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:11.504 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:11.504 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:11.504 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_20 2021-02-20T13:24:11.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:11.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:24:11.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:24:11.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:11.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_20 128M 2021-02-20T13:24:11.506 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:11.506 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:11.506 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:11.506 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:11.506 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_20 2021-02-20T13:24:11.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:11.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:24:11.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:24:11.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:24:11.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:11.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_20 2021-02-20T13:24:11.617 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:24:11.618 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_20 snapshot 2021-02-20T13:24:11.618 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:11.618 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:11.619 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_20 2021-02-20T13:24:11.619 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:24:11.619 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_20 snapshot 2021-02-20T13:24:12.476 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:24:12.497 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:24:12.497 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_20 2021-02-20T13:24:12.498 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:12.498 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:12.499 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_20 2021-02-20T13:24:12.499 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:24:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=7 2021-02-20T13:24:12.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:24:12.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_20 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:24:13.261 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 112830464 bytes 2021-02-20T13:24:13.555 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 134372352 bytes 2021-02-20T13:24:13.817 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 152864768 bytes 2021-02-20T13:24:14.261 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 205059072 bytes 2021-02-20T13:24:14.806 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 277531648 bytes 2021-02-20T13:24:15.171 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 301077504 bytes 2021-02-20T13:24:15.382 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 309336064 bytes 2021-02-20T13:24:15.923 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 371311616 bytes 2021-02-20T13:24:16.104 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 371935232 bytes 2021-02-20T13:24:16.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 385323008 bytes 2021-02-20T13:24:16.971 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 477587456 bytes 2021-02-20T13:24:17.578 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 567724032 bytes 2021-02-20T13:24:17.877 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 586600448 bytes 2021-02-20T13:24:18.305 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 639754240 bytes 2021-02-20T13:24:18.826 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 688616448 bytes 2021-02-20T13:24:19.163 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 721649664 bytes 2021-02-20T13:24:19.502 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:24:19.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:24:19.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:24:19.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:24:19.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_21 2021-02-20T13:24:19.504 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_21 snapshot 128M 2021-02-20T13:24:19.504 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:19.504 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:19.504 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:19.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:19.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_21 2021-02-20T13:24:19.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:19.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:24:19.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:24:19.506 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:19.506 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_21 128M 2021-02-20T13:24:19.506 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:19.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:19.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:19.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:19.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_21 2021-02-20T13:24:19.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:19.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:24:19.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:24:19.509 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:24:19.509 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:19.509 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_21 2021-02-20T13:24:19.926 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:24:19.926 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_21 snapshot 2021-02-20T13:24:19.926 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:19.926 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:19.927 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_21 2021-02-20T13:24:19.927 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:24:19.927 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_21 snapshot 2021-02-20T13:24:20.498 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:24:20.526 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:24:20.526 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_21 2021-02-20T13:24:20.526 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:20.527 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:20.527 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_21 2021-02-20T13:24:20.527 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:24:20.528 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=6 2021-02-20T13:24:20.528 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:24:20.528 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_21 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:24:20.964 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 40633344 bytes 2021-02-20T13:24:21.399 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 95567872 bytes 2021-02-20T13:24:21.768 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 137435136 bytes 2021-02-20T13:24:22.163 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 187032576 bytes 2021-02-20T13:24:22.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 224022528 bytes 2021-02-20T13:24:22.897 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 260832256 bytes 2021-02-20T13:24:23.308 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 304693248 bytes 2021-02-20T13:24:23.825 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 378097664 bytes 2021-02-20T13:24:24.312 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 432145408 bytes 2021-02-20T13:24:24.737 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 487702528 bytes 2021-02-20T13:24:25.265 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 537466880 bytes 2021-02-20T13:24:25.550 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 556624896 bytes 2021-02-20T13:24:25.863 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 584750080 bytes 2021-02-20T13:24:26.242 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 624313344 bytes 2021-02-20T13:24:26.420 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 625458176 bytes 2021-02-20T13:24:26.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:24:26.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:24:26.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:24:26.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:24:26.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_22 2021-02-20T13:24:26.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_22 snapshot 128M 2021-02-20T13:24:26.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:26.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:26.534 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:26.534 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:26.534 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_22 2021-02-20T13:24:26.534 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:26.534 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:24:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:24:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_22 128M 2021-02-20T13:24:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:26.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:26.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:26.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_22 2021-02-20T13:24:26.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:26.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:24:26.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:24:26.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:24:26.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:26.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_22 2021-02-20T13:24:26.595 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:24:26.595 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_22 snapshot 2021-02-20T13:24:26.595 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:26.595 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:26.596 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_22 2021-02-20T13:24:26.596 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:24:26.596 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_22 snapshot 2021-02-20T13:24:27.319 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:24:27.336 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:24:27.336 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_22 2021-02-20T13:24:27.337 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:27.337 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:27.337 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_22 2021-02-20T13:24:27.337 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:24:27.338 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=5 2021-02-20T13:24:27.338 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:24:27.338 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_22 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:24:27.592 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 617472 bytes 2021-02-20T13:24:27.898 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 34440192 bytes 2021-02-20T13:24:28.227 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 59529216 bytes 2021-02-20T13:24:28.716 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 110779392 bytes 2021-02-20T13:24:28.971 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 122470400 bytes 2021-02-20T13:24:29.421 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 186407936 bytes 2021-02-20T13:24:29.862 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 213494784 bytes 2021-02-20T13:24:30.477 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 277466112 bytes 2021-02-20T13:24:30.803 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 305698816 bytes 2021-02-20T13:24:31.063 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 321761280 bytes 2021-02-20T13:24:31.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 328189952 bytes 2021-02-20T13:24:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 336733184 bytes 2021-02-20T13:24:32.026 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 422312960 bytes 2021-02-20T13:24:32.343 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:24:32.344 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:24:32.344 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:24:32.344 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:24:32.344 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_23 2021-02-20T13:24:32.345 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_23 snapshot 128M 2021-02-20T13:24:32.345 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:32.345 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:32.345 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:32.345 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:32.346 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_23 2021-02-20T13:24:32.346 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:32.346 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:24:32.346 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:24:32.346 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:32.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_23 128M 2021-02-20T13:24:32.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:32.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:32.347 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:32.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:32.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_23 2021-02-20T13:24:32.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:32.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:24:32.348 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:24:32.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:24:32.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:32.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_23 2021-02-20T13:24:32.436 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:24:32.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:24:32.437 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T13:24:32.438 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:24:32.438 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_23 snapshot 2021-02-20T13:24:32.438 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:32.438 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:32.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_23 2021-02-20T13:24:32.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:24:32.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_23 snapshot 2021-02-20T13:24:32.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T13:24:32.433+0000 7f207b3b8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 25487) UID: 1000 2021-02-20T13:24:32.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:24:33.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:24:33.514 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:24:33.515 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_23 2021-02-20T13:24:33.515 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:33.515 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:33.515 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_23 2021-02-20T13:24:33.516 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:24:33.516 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=5 2021-02-20T13:24:33.517 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:24:33.517 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_23 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:24:33.877 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 28485632 bytes 2021-02-20T13:24:34.165 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 51217408 bytes 2021-02-20T13:24:34.438 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:24:34.439 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:24:34.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:24:34.440 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:24:34.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:24:34.441 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:24:34.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:24:34.446 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T13:24:34.705 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 124028928 bytes 2021-02-20T13:24:35.153 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 173963264 bytes 2021-02-20T13:24:35.595 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 242171904 bytes 2021-02-20T13:24:36.116 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 303962112 bytes 2021-02-20T13:24:36.379 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 305781760 bytes 2021-02-20T13:24:36.928 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 378429440 bytes 2021-02-20T13:24:37.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 467918848 bytes 2021-02-20T13:24:38.012 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 526318592 bytes 2021-02-20T13:24:38.478 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 592530432 bytes 2021-02-20T13:24:38.521 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:24:38.522 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:24:38.522 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:24:38.523 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:24:38.523 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_24 2021-02-20T13:24:38.523 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_24 snapshot 128M 2021-02-20T13:24:38.524 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:38.524 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:38.525 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:38.525 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:38.525 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_24 2021-02-20T13:24:38.526 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:38.526 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:24:38.527 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:24:38.527 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:38.527 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_24 128M 2021-02-20T13:24:38.528 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:38.528 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:38.529 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:38.529 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:38.529 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_24 2021-02-20T13:24:38.530 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:38.530 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:24:38.531 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:24:38.531 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:24:38.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:38.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_24 2021-02-20T13:24:38.591 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:24:38.591 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_24 snapshot 2021-02-20T13:24:38.592 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:38.592 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:38.592 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_24 2021-02-20T13:24:38.593 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:24:38.593 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_24 snapshot 2021-02-20T13:24:39.338 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:24:39.368 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:24:39.368 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_24 2021-02-20T13:24:39.369 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:39.369 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:39.369 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_24 2021-02-20T13:24:39.370 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:24:39.371 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=13 2021-02-20T13:24:39.371 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:24:39.371 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_24 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:24:39.761 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 32025600 bytes 2021-02-20T13:24:40.020 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 69444608 bytes 2021-02-20T13:24:40.377 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 84497408 bytes 2021-02-20T13:24:40.699 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 113784832 bytes 2021-02-20T13:24:41.263 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 166004736 bytes 2021-02-20T13:24:41.596 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 190639104 bytes 2021-02-20T13:24:42.047 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 242793472 bytes 2021-02-20T13:24:42.381 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 269448192 bytes 2021-02-20T13:24:42.795 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 317091840 bytes 2021-02-20T13:24:43.216 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 359897088 bytes 2021-02-20T13:24:43.838 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 440986624 bytes 2021-02-20T13:24:44.194 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 484346880 bytes 2021-02-20T13:24:44.552 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 484964352 bytes 2021-02-20T13:24:44.785 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 495015936 bytes 2021-02-20T13:24:45.436 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 588205056 bytes 2021-02-20T13:24:45.971 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 662276096 bytes 2021-02-20T13:24:46.402 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 710481920 bytes 2021-02-20T13:24:46.670 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 730639360 bytes 2021-02-20T13:24:46.876 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 731250688 bytes 2021-02-20T13:24:47.405 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 808441856 bytes 2021-02-20T13:24:47.800 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 841251840 bytes 2021-02-20T13:24:48.095 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 864664576 bytes 2021-02-20T13:24:48.426 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 903901184 bytes 2021-02-20T13:24:49.025 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 972149760 bytes 2021-02-20T13:24:49.372 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 1008603136 bytes 2021-02-20T13:24:49.892 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 1070539776 bytes 2021-02-20T13:24:50.566 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 1132449792 bytes 2021-02-20T13:24:50.936 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2800 IOs, 1166957568 bytes 2021-02-20T13:24:51.149 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2900 IOs, 1174994944 bytes 2021-02-20T13:24:51.512 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3000 IOs, 1209283584 bytes 2021-02-20T13:24:52.108 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3100 IOs, 1294580736 bytes 2021-02-20T13:24:52.375 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:24:52.375 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:24:52.375 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:24:52.375 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:24:52.376 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_25 2021-02-20T13:24:52.376 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_25 snapshot 128M 2021-02-20T13:24:52.376 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:52.376 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:52.376 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:52.377 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:52.377 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_25 2021-02-20T13:24:52.377 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:52.377 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:24:52.378 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:24:52.378 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:52.378 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_25 128M 2021-02-20T13:24:52.378 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:52.379 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:52.379 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:52.379 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:52.379 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_25 2021-02-20T13:24:52.379 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:52.380 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:24:52.380 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:24:52.380 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:24:52.380 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:24:52.380 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_25 2021-02-20T13:24:52.437 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:24:52.438 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_25 snapshot 2021-02-20T13:24:52.438 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:52.438 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:52.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_25 2021-02-20T13:24:52.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:24:52.439 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_25 snapshot 2021-02-20T13:24:53.515 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:24:53.542 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:24:53.542 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_25 2021-02-20T13:24:53.542 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:24:53.542 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:24:53.543 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_25 2021-02-20T13:24:53.543 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:24:53.545 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=12 2021-02-20T13:24:53.545 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:24:53.545 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_25 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:24:53.976 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 37893120 bytes 2021-02-20T13:24:54.213 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 57101312 bytes 2021-02-20T13:24:54.586 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 92191744 bytes 2021-02-20T13:24:54.882 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 104407040 bytes 2021-02-20T13:24:55.218 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 123028480 bytes 2021-02-20T13:24:55.647 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 193128448 bytes 2021-02-20T13:24:56.211 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T13:24:56.209+0000 7f03ccd38700 -1 reset not still connected to 0x556b8794fc20 2021-02-20T13:24:56.211 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T13:24:56.209+0000 7f03ccd38700 -1 reset not still connected to 0x556b93879400 2021-02-20T13:24:56.211 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T13:24:56.209+0000 7f03ccd38700 -1 reset not still connected to 0x556b9f07cf70 2021-02-20T13:24:56.212 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T13:24:56.209+0000 7f03ccd38700 -1 reset not still connected to 0x556b9f07f670 2021-02-20T13:24:56.336 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 285425664 bytes 2021-02-20T13:24:56.946 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 364957696 bytes 2021-02-20T13:24:57.293 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 400254976 bytes 2021-02-20T13:24:57.673 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 454781952 bytes 2021-02-20T13:24:57.922 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 455371776 bytes 2021-02-20T13:24:58.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 531411968 bytes 2021-02-20T13:24:58.699 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 537841664 bytes 2021-02-20T13:24:59.039 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 577741824 bytes 2021-02-20T13:24:59.522 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 623118336 bytes 2021-02-20T13:24:59.917 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 669487104 bytes 2021-02-20T13:25:00.432 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 721059840 bytes 2021-02-20T13:25:00.655 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 731788288 bytes 2021-02-20T13:25:00.963 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 756009984 bytes 2021-02-20T13:25:01.718 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 860436480 bytes 2021-02-20T13:25:01.979 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 877674496 bytes 2021-02-20T13:25:02.675 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 978828288 bytes 2021-02-20T13:25:03.044 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 1016998912 bytes 2021-02-20T13:25:03.456 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 1060312064 bytes 2021-02-20T13:25:03.863 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 1101592576 bytes 2021-02-20T13:25:04.102 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 1106574336 bytes 2021-02-20T13:25:04.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 1153963008 bytes 2021-02-20T13:25:05.210 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2800 IOs, 1196567552 bytes 2021-02-20T13:25:05.551 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:25:05.551 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:25:05.551 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:25:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:25:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_26 2021-02-20T13:25:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_26 snapshot 128M 2021-02-20T13:25:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:25:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:05.553 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:25:05.553 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:05.553 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_26 2021-02-20T13:25:05.553 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:05.554 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:25:05.554 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:25:05.554 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:05.554 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_26 128M 2021-02-20T13:25:05.554 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:25:05.555 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:05.555 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:25:05.555 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:05.555 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_26 2021-02-20T13:25:05.556 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:05.556 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:25:05.556 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:25:05.556 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:25:05.556 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:05.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_26 2021-02-20T13:25:05.618 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:25:05.618 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_26 snapshot 2021-02-20T13:25:05.618 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:25:05.619 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:25:05.619 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_26 2021-02-20T13:25:05.619 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:25:05.619 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_26 snapshot 2021-02-20T13:25:06.335 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:25:06.351 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:25:06.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_26 2021-02-20T13:25:06.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:25:06.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:25:06.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_26 2021-02-20T13:25:06.353 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:25:06.353 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=14 2021-02-20T13:25:06.353 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:25:06.354 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_26 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:25:06.762 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 39746560 bytes 2021-02-20T13:25:06.975 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 44026880 bytes 2021-02-20T13:25:07.335 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 87781376 bytes 2021-02-20T13:25:07.775 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 119763968 bytes 2021-02-20T13:25:08.212 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 178791424 bytes 2021-02-20T13:25:08.596 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 221372416 bytes 2021-02-20T13:25:08.971 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 263648256 bytes 2021-02-20T13:25:09.285 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 286922752 bytes 2021-02-20T13:25:09.698 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 344802304 bytes 2021-02-20T13:25:10.122 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 385783808 bytes 2021-02-20T13:25:10.562 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 434228224 bytes 2021-02-20T13:25:10.798 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 444511232 bytes 2021-02-20T13:25:11.229 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 496055296 bytes 2021-02-20T13:25:11.639 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 545368064 bytes 2021-02-20T13:25:12.161 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 609908736 bytes 2021-02-20T13:25:12.711 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 687191040 bytes 2021-02-20T13:25:13.245 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 740526080 bytes 2021-02-20T13:25:13.618 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 776523776 bytes 2021-02-20T13:25:13.854 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 786906112 bytes 2021-02-20T13:25:14.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 855673856 bytes 2021-02-20T13:25:14.693 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 902276096 bytes 2021-02-20T13:25:15.014 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 913339392 bytes 2021-02-20T13:25:15.257 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 926057472 bytes 2021-02-20T13:25:15.734 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 986796032 bytes 2021-02-20T13:25:16.375 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 1058088960 bytes 2021-02-20T13:25:16.963 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 1140214784 bytes 2021-02-20T13:25:17.709 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 1248369664 bytes 2021-02-20T13:25:18.118 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2800 IOs, 1296020480 bytes 2021-02-20T13:25:18.407 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2900 IOs, 1312697344 bytes 2021-02-20T13:25:18.743 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3000 IOs, 1341124608 bytes 2021-02-20T13:25:19.037 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3100 IOs, 1377648640 bytes 2021-02-20T13:25:19.685 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3200 IOs, 1450785792 bytes 2021-02-20T13:25:20.265 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3300 IOs, 1547173888 bytes 2021-02-20T13:25:20.358 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:25:20.359 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:25:20.359 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:25:20.359 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:25:20.360 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_27 2021-02-20T13:25:20.360 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_27 snapshot 128M 2021-02-20T13:25:20.360 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:25:20.360 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:20.360 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:25:20.361 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:20.361 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_27 2021-02-20T13:25:20.361 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:20.361 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:25:20.362 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:25:20.362 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:20.362 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_27 128M 2021-02-20T13:25:20.363 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:25:20.363 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:20.363 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:25:20.363 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:20.364 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_27 2021-02-20T13:25:20.364 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:20.364 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:25:20.364 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:25:20.364 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:25:20.365 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:20.365 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_27 2021-02-20T13:25:20.427 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:25:20.427 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_27 snapshot 2021-02-20T13:25:20.428 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:25:20.428 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:25:20.428 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_27 2021-02-20T13:25:20.428 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:25:20.428 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_27 snapshot 2021-02-20T13:25:21.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:25:21.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:25:21.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_27 2021-02-20T13:25:21.534 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:25:21.534 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:25:21.534 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_27 2021-02-20T13:25:21.534 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:25:21.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=11 2021-02-20T13:25:21.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:25:21.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_27 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:25:21.827 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 609280 bytes 2021-02-20T13:25:22.433 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 97625088 bytes 2021-02-20T13:25:22.833 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 120745984 bytes 2021-02-20T13:25:23.368 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 177704960 bytes 2021-02-20T13:25:23.694 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 189424640 bytes 2021-02-20T13:25:24.248 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 261229568 bytes 2021-02-20T13:25:24.366 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5754999794651394) > weight (0.25) 2021-02-20T13:25:24.366 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5051207982293453) > weight (0.25) 2021-02-20T13:25:24.367 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T13:25:24.367 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T13:25:24.367 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T13:25:24.368 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T13:25:24.368+0000 7fdf4a66a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 22174) UID: 1000 2021-02-20T13:25:24.642 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 296823808 bytes 2021-02-20T13:25:24.952 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 314412032 bytes 2021-02-20T13:25:24.970 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:25:25.144 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 316427264 bytes 2021-02-20T13:25:25.369 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T13:25:25.369 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:25:25.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:25:25.371 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T13:25:25.371 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T13:25:25.372 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T13:25:25.375 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T13:25:25.375 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T13:25:25.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 346782720 bytes 2021-02-20T13:25:25.832 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 368615424 bytes 2021-02-20T13:25:26.106 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 384612352 bytes 2021-02-20T13:25:26.703 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 452074496 bytes 2021-02-20T13:25:27.229 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 480035840 bytes 2021-02-20T13:25:27.564 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 509156352 bytes 2021-02-20T13:25:28.240 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 585156608 bytes 2021-02-20T13:25:28.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 610578432 bytes 2021-02-20T13:25:28.933 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 657032192 bytes 2021-02-20T13:25:29.451 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 717637632 bytes 2021-02-20T13:25:29.881 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 755187712 bytes 2021-02-20T13:25:30.407 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 822340608 bytes 2021-02-20T13:25:30.707 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 841292800 bytes 2021-02-20T13:25:31.150 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 885110784 bytes 2021-02-20T13:25:31.560 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 928109568 bytes 2021-02-20T13:25:32.134 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 1003782144 bytes 2021-02-20T13:25:32.540 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:25:32.541 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:25:32.541 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:25:32.541 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:25:32.541 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_28 2021-02-20T13:25:32.542 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_28 snapshot 128M 2021-02-20T13:25:32.542 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:25:32.542 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:32.542 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:25:32.543 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:32.543 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_28 2021-02-20T13:25:32.543 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:32.543 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:25:32.543 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:25:32.544 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:32.544 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_28 128M 2021-02-20T13:25:32.544 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:25:32.544 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:32.544 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:25:32.545 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:32.545 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_28 2021-02-20T13:25:32.545 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:32.545 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:25:32.546 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:25:32.546 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:25:32.546 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:32.546 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_28 2021-02-20T13:25:32.604 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:25:32.604 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_28 snapshot 2021-02-20T13:25:32.605 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:25:32.605 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:25:32.605 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_28 2021-02-20T13:25:32.605 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:25:32.606 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_28 snapshot 2021-02-20T13:25:33.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:25:33.524 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:25:33.524 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_28 2021-02-20T13:25:33.525 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:25:33.525 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:25:33.525 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_28 2021-02-20T13:25:33.526 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:25:33.526 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=13 2021-02-20T13:25:33.527 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:25:33.527 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_28 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:25:34.263 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 76746752 bytes 2021-02-20T13:25:34.702 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 137956352 bytes 2021-02-20T13:25:35.047 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 172987392 bytes 2021-02-20T13:25:35.315 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 188995584 bytes 2021-02-20T13:25:35.797 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 255706112 bytes 2021-02-20T13:25:36.137 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 277764096 bytes 2021-02-20T13:25:36.508 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 317870080 bytes 2021-02-20T13:25:36.762 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 327292928 bytes 2021-02-20T13:25:37.124 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 360145920 bytes 2021-02-20T13:25:37.328 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 381465600 bytes 2021-02-20T13:25:37.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 386548736 bytes 2021-02-20T13:25:38.028 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 407709696 bytes 2021-02-20T13:25:38.359 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 445768704 bytes 2021-02-20T13:25:38.911 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 510194688 bytes 2021-02-20T13:25:39.233 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 536214528 bytes 2021-02-20T13:25:39.688 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 578827264 bytes 2021-02-20T13:25:40.351 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 658233344 bytes 2021-02-20T13:25:40.792 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 726844416 bytes 2021-02-20T13:25:41.129 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 737610752 bytes 2021-02-20T13:25:41.456 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 766811136 bytes 2021-02-20T13:25:41.937 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 821349376 bytes 2021-02-20T13:25:42.482 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 894648320 bytes 2021-02-20T13:25:42.753 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 908702720 bytes 2021-02-20T13:25:43.040 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 924975104 bytes 2021-02-20T13:25:43.520 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 994266112 bytes 2021-02-20T13:25:43.834 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 1017789440 bytes 2021-02-20T13:25:44.257 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 1071475712 bytes 2021-02-20T13:25:44.579 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2800 IOs, 1089751040 bytes 2021-02-20T13:25:44.776 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2900 IOs, 1090347008 bytes 2021-02-20T13:25:45.199 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3000 IOs, 1136237568 bytes 2021-02-20T13:25:45.511 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3100 IOs, 1159283712 bytes 2021-02-20T13:25:45.987 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3200 IOs, 1215706112 bytes 2021-02-20T13:25:46.262 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3300 IOs, 1258438656 bytes 2021-02-20T13:25:46.534 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:25:46.534 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:25:46.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:25:46.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:25:46.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_29 2021-02-20T13:25:46.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_29 snapshot 128M 2021-02-20T13:25:46.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:25:46.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:46.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:25:46.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:46.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_29 2021-02-20T13:25:46.540 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:46.540 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:25:46.543 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:25:46.543 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:46.543 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_29 128M 2021-02-20T13:25:46.544 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:25:46.544 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:46.545 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:25:46.545 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:46.545 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_29 2021-02-20T13:25:46.546 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:46.546 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:25:46.547 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:25:46.547 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:25:46.548 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:46.548 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_29 2021-02-20T13:25:46.656 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:25:46.657 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_29 snapshot 2021-02-20T13:25:46.657 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:25:46.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:25:46.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_29 2021-02-20T13:25:46.659 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:25:46.659 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_29 snapshot 2021-02-20T13:25:47.365 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:25:47.384 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:25:47.385 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_29 2021-02-20T13:25:47.385 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:25:47.385 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:25:47.385 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_29 2021-02-20T13:25:47.386 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:25:47.386 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=9 2021-02-20T13:25:47.387 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:25:47.387 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_29 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:25:48.114 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 67270656 bytes 2021-02-20T13:25:48.525 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 113889280 bytes 2021-02-20T13:25:48.986 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 167675904 bytes 2021-02-20T13:25:49.309 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 192077824 bytes 2021-02-20T13:25:49.592 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 222586880 bytes 2021-02-20T13:25:49.979 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 245760000 bytes 2021-02-20T13:25:50.346 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 267189248 bytes 2021-02-20T13:25:50.890 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 320684032 bytes 2021-02-20T13:25:51.273 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 359209984 bytes 2021-02-20T13:25:51.660 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 398457856 bytes 2021-02-20T13:25:52.036 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 435819520 bytes 2021-02-20T13:25:52.346 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 457949184 bytes 2021-02-20T13:25:52.664 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 486076416 bytes 2021-02-20T13:25:53.210 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 517544960 bytes 2021-02-20T13:25:53.438 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 519636992 bytes 2021-02-20T13:25:53.759 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 520235008 bytes 2021-02-20T13:25:54.449 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 617517056 bytes 2021-02-20T13:25:55.078 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 708134912 bytes 2021-02-20T13:25:55.952 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 843770880 bytes 2021-02-20T13:25:56.301 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 856525824 bytes 2021-02-20T13:25:56.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:25:56.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:25:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:25:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:25:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_30 2021-02-20T13:25:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_30 snapshot 128M 2021-02-20T13:25:56.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:25:56.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:56.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:25:56.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:56.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_30 2021-02-20T13:25:56.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:56.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:25:56.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:25:56.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:56.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_30 128M 2021-02-20T13:25:56.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:25:56.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:56.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:25:56.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:56.397 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_30 2021-02-20T13:25:56.397 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:56.397 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:25:56.397 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:25:56.398 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:25:56.398 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:25:56.398 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_30 2021-02-20T13:25:56.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:25:56.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_30 snapshot 2021-02-20T13:25:56.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:25:56.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:25:56.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_30 2021-02-20T13:25:56.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:25:56.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_30 snapshot 2021-02-20T13:25:57.362 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:25:57.381 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:25:57.381 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_30 2021-02-20T13:25:57.382 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:25:57.382 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:25:57.382 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_30 2021-02-20T13:25:57.383 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:25:57.383 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=14 2021-02-20T13:25:57.384 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:25:57.384 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_30 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:25:57.673 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 11976704 bytes 2021-02-20T13:25:57.940 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 24656896 bytes 2021-02-20T13:25:58.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 64424960 bytes 2021-02-20T13:25:58.738 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 79401984 bytes 2021-02-20T13:25:59.013 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 87268352 bytes 2021-02-20T13:25:59.226 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 102042624 bytes 2021-02-20T13:25:59.578 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 120650752 bytes 2021-02-20T13:25:59.956 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 162676736 bytes 2021-02-20T13:26:00.258 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 206383104 bytes 2021-02-20T13:26:00.734 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 259978240 bytes 2021-02-20T13:26:01.333 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 311469056 bytes 2021-02-20T13:26:01.663 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 338394112 bytes 2021-02-20T13:26:01.862 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 339025920 bytes 2021-02-20T13:26:02.307 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 398898176 bytes 2021-02-20T13:26:02.806 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 445281280 bytes 2021-02-20T13:26:03.196 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 477176832 bytes 2021-02-20T13:26:03.575 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 513592320 bytes 2021-02-20T13:26:04.078 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 588156928 bytes 2021-02-20T13:26:04.360 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 589159424 bytes 2021-02-20T13:26:05.087 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 685147136 bytes 2021-02-20T13:26:05.390 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 705219584 bytes 2021-02-20T13:26:05.581 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 705843200 bytes 2021-02-20T13:26:05.937 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T13:26:05.931+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a3b835f0 2021-02-20T13:26:05.937 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T13:26:05.931+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a44b0410 2021-02-20T13:26:05.937 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T13:26:05.931+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a44b3a80 2021-02-20T13:26:05.938 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T13:26:05.931+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a5281e90 2021-02-20T13:26:05.938 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T13:26:05.931+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a5678f20 2021-02-20T13:26:05.938 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T13:26:05.931+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a600c680 2021-02-20T13:26:05.938 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T13:26:05.931+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6abc0be90 2021-02-20T13:26:05.939 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T13:26:05.931+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ac75be10 2021-02-20T13:26:05.939 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T13:26:05.935+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6c3924000 2021-02-20T13:26:05.939 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T13:26:05.935+0000 7fcbd16b5700 -1 reset not still connected to 0x55c69ae76ea0 2021-02-20T13:26:05.939 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T13:26:05.935+0000 7fcbd16b5700 -1 reset not still connected to 0x55c69ae77040 2021-02-20T13:26:05.939 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T13:26:05.935+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a5281e90 2021-02-20T13:26:05.940 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T13:26:05.935+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6c3924000 2021-02-20T13:26:06.191 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 776776704 bytes 2021-02-20T13:26:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 813422592 bytes 2021-02-20T13:26:06.977 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 814041088 bytes 2021-02-20T13:26:07.372 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 857225216 bytes 2021-02-20T13:26:07.721 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 889817088 bytes 2021-02-20T13:26:07.937 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2800 IOs, 895559680 bytes 2021-02-20T13:26:08.237 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2900 IOs, 938127360 bytes 2021-02-20T13:26:08.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3000 IOs, 938728448 bytes 2021-02-20T13:26:08.915 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3100 IOs, 976161792 bytes 2021-02-20T13:26:09.382 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3200 IOs, 1029721088 bytes 2021-02-20T13:26:09.716 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3300 IOs, 1074023424 bytes 2021-02-20T13:26:10.378 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3400 IOs, 1153531904 bytes 2021-02-20T13:26:10.586 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3500 IOs, 1154140160 bytes 2021-02-20T13:26:10.784 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3600 IOs, 1154759680 bytes 2021-02-20T13:26:11.064 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3700 IOs, 1178256384 bytes 2021-02-20T13:26:11.384 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3800 IOs, 1198974976 bytes 2021-02-20T13:26:11.388 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:26:11.388 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:26:11.389 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:26:11.389 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:26:11.389 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_31 2021-02-20T13:26:11.389 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_31 snapshot 128M 2021-02-20T13:26:11.390 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:26:11.390 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:11.390 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:26:11.391 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:11.391 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_31 2021-02-20T13:26:11.391 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:11.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:26:11.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:26:11.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:11.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_31 128M 2021-02-20T13:26:11.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:26:11.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:11.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:26:11.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:11.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_31 2021-02-20T13:26:11.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:11.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:26:11.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:26:11.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:26:11.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:11.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_31 2021-02-20T13:26:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:26:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_31 snapshot 2021-02-20T13:26:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:26:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:26:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_31 2021-02-20T13:26:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:26:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_31 snapshot 2021-02-20T13:26:12.363 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:26:12.387 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:26:12.387 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_31 2021-02-20T13:26:12.388 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:26:12.388 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:26:12.388 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_31 2021-02-20T13:26:12.388 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:26:12.389 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=10 2021-02-20T13:26:12.389 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:26:12.389 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_31 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:26:12.764 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 29232128 bytes 2021-02-20T13:26:13.138 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 63731712 bytes 2021-02-20T13:26:13.708 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 112685056 bytes 2021-02-20T13:26:14.207 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 173208576 bytes 2021-02-20T13:26:14.549 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 190392320 bytes 2021-02-20T13:26:14.938 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 228495360 bytes 2021-02-20T13:26:15.329 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 271837184 bytes 2021-02-20T13:26:15.750 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 309624832 bytes 2021-02-20T13:26:16.217 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 367235072 bytes 2021-02-20T13:26:16.852 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 445137920 bytes 2021-02-20T13:26:17.482 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 530049024 bytes 2021-02-20T13:26:17.849 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 562216960 bytes 2021-02-20T13:26:18.466 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 613504000 bytes 2021-02-20T13:26:18.993 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 674763776 bytes 2021-02-20T13:26:19.333 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 708258816 bytes 2021-02-20T13:26:19.931 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 776484864 bytes 2021-02-20T13:26:20.260 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 825075712 bytes 2021-02-20T13:26:20.871 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 881107968 bytes 2021-02-20T13:26:21.185 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 908234752 bytes 2021-02-20T13:26:21.601 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 951864320 bytes 2021-02-20T13:26:21.970 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 987216896 bytes 2021-02-20T13:26:22.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:26:22.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:26:22.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:26:22.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:26:22.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_32 2021-02-20T13:26:22.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_32 snapshot 128M 2021-02-20T13:26:22.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:26:22.397 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:22.397 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:26:22.397 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:22.397 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_32 2021-02-20T13:26:22.397 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:22.398 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:26:22.398 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:26:22.398 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:22.398 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_32 128M 2021-02-20T13:26:22.398 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:26:22.399 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:22.399 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:26:22.399 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:22.399 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_32 2021-02-20T13:26:22.400 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:22.400 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:26:22.400 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:26:22.400 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:26:22.400 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:22.401 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_32 2021-02-20T13:26:22.459 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:26:22.459 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_32 snapshot 2021-02-20T13:26:22.459 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:26:22.460 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:26:22.460 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_32 2021-02-20T13:26:22.460 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:26:22.460 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_32 snapshot 2021-02-20T13:26:22.658 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:26:22.679 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:26:22.679 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_32 2021-02-20T13:26:22.679 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:26:22.680 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:26:22.680 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_32 2021-02-20T13:26:22.680 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:26:22.681 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=8 2021-02-20T13:26:22.681 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:26:22.682 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_32 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:26:22.936 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 612352 bytes 2021-02-20T13:26:23.190 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 19088384 bytes 2021-02-20T13:26:23.608 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 65996800 bytes 2021-02-20T13:26:23.800 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 67490816 bytes 2021-02-20T13:26:24.116 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 104924160 bytes 2021-02-20T13:26:24.403 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 114404352 bytes 2021-02-20T13:26:24.806 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 164396032 bytes 2021-02-20T13:26:25.301 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 215429120 bytes 2021-02-20T13:26:26.078 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 298978304 bytes 2021-02-20T13:26:26.411 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 318406656 bytes 2021-02-20T13:26:26.855 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 388006912 bytes 2021-02-20T13:26:27.268 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 410797056 bytes 2021-02-20T13:26:27.798 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 477890560 bytes 2021-02-20T13:26:28.271 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 515291136 bytes 2021-02-20T13:26:28.828 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 543782912 bytes 2021-02-20T13:26:29.447 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.576185052540282) > weight (0.25) 2021-02-20T13:26:29.449 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5788479229993152) > weight (0.25) 2021-02-20T13:26:29.449 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T13:26:29.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T13:26:29.450 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T13:26:29.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T13:26:29.450+0000 7f8193753700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 27849) UID: 1000 2021-02-20T13:26:29.531 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 665961472 bytes 2021-02-20T13:26:29.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:26:30.057 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 714588160 bytes 2021-02-20T13:26:30.399 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 739357696 bytes 2021-02-20T13:26:30.604 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 739955712 bytes 2021-02-20T13:26:30.686 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:26:30.686 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:26:30.687 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:26:30.687 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:26:30.688 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_33 2021-02-20T13:26:30.688 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_33 snapshot 128M 2021-02-20T13:26:30.689 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:26:30.689 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:30.690 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:26:30.690 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:30.690 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_33 2021-02-20T13:26:30.690 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:30.691 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:26:30.691 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:26:30.691 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:30.692 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_33 128M 2021-02-20T13:26:30.692 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:26:30.692 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:30.693 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:26:30.693 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:30.693 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_33 2021-02-20T13:26:30.693 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:30.693 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:26:30.694 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:26:30.694 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:26:30.694 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:30.694 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_33 2021-02-20T13:26:30.751 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:26:30.753 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_33 snapshot 2021-02-20T13:26:30.754 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:26:30.754 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:26:30.755 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_33 2021-02-20T13:26:30.755 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:26:30.756 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_33 snapshot 2021-02-20T13:26:31.372 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:26:31.391 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:26:31.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_33 2021-02-20T13:26:31.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:26:31.392 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:26:31.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_33 2021-02-20T13:26:31.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:26:31.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=10 2021-02-20T13:26:31.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:26:31.394 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_33 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:26:31.939 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 61006848 bytes 2021-02-20T13:26:32.160 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 70606848 bytes 2021-02-20T13:26:32.353 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 71218176 bytes 2021-02-20T13:26:32.451 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T13:26:32.453 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:26:32.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:26:32.454 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T13:26:32.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T13:26:32.454 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T13:26:32.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T13:26:32.457 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T13:26:32.687 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 107744256 bytes 2021-02-20T13:26:33.247 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 181865472 bytes 2021-02-20T13:26:33.515 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T13:26:33.474+0000 7f4a1bb72700 -1 reset not still connected to 0x55fc710a8be0 2021-02-20T13:26:33.515 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T13:26:33.474+0000 7f4a1bb72700 -1 reset not still connected to 0x55fc75f48ff0 2021-02-20T13:26:33.516 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T13:26:33.474+0000 7f4a1bb72700 -1 reset not still connected to 0x55fc88554680 2021-02-20T13:26:33.516 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T13:26:33.474+0000 7f4a1bb72700 -1 reset not still connected to 0x55fc8e5fb5f0 2021-02-20T13:26:33.516 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T13:26:33.474+0000 7f4a1bb72700 -1 reset not still connected to 0x55fc913f4410 2021-02-20T13:26:33.516 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T13:26:33.474+0000 7f4a1bb72700 -1 reset not still connected to 0x55fc92d72410 2021-02-20T13:26:33.734 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 244447232 bytes 2021-02-20T13:26:34.232 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 295064576 bytes 2021-02-20T13:26:34.547 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 319629312 bytes 2021-02-20T13:26:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 358753280 bytes 2021-02-20T13:26:35.376 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 417884160 bytes 2021-02-20T13:26:35.581 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 418498560 bytes 2021-02-20T13:26:35.977 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 437986304 bytes 2021-02-20T13:26:37.149 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 580853760 bytes 2021-02-20T13:26:37.951 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 655323136 bytes 2021-02-20T13:26:38.418 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 678939648 bytes 2021-02-20T13:26:39.255 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 757675008 bytes 2021-02-20T13:26:39.810 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 786141184 bytes 2021-02-20T13:26:40.219 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 802871296 bytes 2021-02-20T13:26:40.660 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 813001728 bytes 2021-02-20T13:26:41.361 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 866695168 bytes 2021-02-20T13:26:41.398 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:26:41.399 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:26:41.400 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:26:41.400 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:26:41.401 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_34 2021-02-20T13:26:41.402 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_34 snapshot 128M 2021-02-20T13:26:41.402 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:26:41.402 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:41.403 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:26:41.403 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:41.404 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_34 2021-02-20T13:26:41.404 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:41.404 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:26:41.405 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:26:41.405 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:41.406 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_34 128M 2021-02-20T13:26:41.410 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:26:41.410 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:41.411 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:26:41.411 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:41.412 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_34 2021-02-20T13:26:41.413 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:41.414 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:26:41.414 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:26:41.415 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:26:41.415 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:41.416 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_34 2021-02-20T13:26:41.489 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:26:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_34 snapshot 2021-02-20T13:26:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:26:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:26:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_34 2021-02-20T13:26:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:26:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_34 snapshot 2021-02-20T13:26:42.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:26:42.527 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:26:42.527 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_34 2021-02-20T13:26:42.527 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:26:42.528 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:26:42.528 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_34 2021-02-20T13:26:42.528 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:26:42.529 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=10 2021-02-20T13:26:42.529 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:26:42.529 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_34 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:26:43.008 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 46586880 bytes 2021-02-20T13:26:43.344 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 75294720 bytes 2021-02-20T13:26:43.845 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 97298432 bytes 2021-02-20T13:26:44.272 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 128327680 bytes 2021-02-20T13:26:44.376 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:26:44.379 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:26:44.379 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T13:26:44.380 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T13:26:44.374+0000 7f1fbeb6c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 29060) UID: 1000 2021-02-20T13:26:44.544 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 128932864 bytes 2021-02-20T13:26:44.780 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:26:44.955 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 160793600 bytes 2021-02-20T13:26:45.410 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 188024832 bytes 2021-02-20T13:26:45.866 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 229730304 bytes 2021-02-20T13:26:46.632 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 319339520 bytes 2021-02-20T13:26:47.360 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 417665024 bytes 2021-02-20T13:26:47.834 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 439665664 bytes 2021-02-20T13:26:48.289 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 472810496 bytes 2021-02-20T13:26:48.780 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 496582656 bytes 2021-02-20T13:26:49.101 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 518864896 bytes 2021-02-20T13:26:49.541 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 535940096 bytes 2021-02-20T13:26:50.133 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 601467904 bytes 2021-02-20T13:26:50.740 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 642734080 bytes 2021-02-20T13:26:51.267 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 690440192 bytes 2021-02-20T13:26:51.742 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 728966144 bytes 2021-02-20T13:26:52.265 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 771799040 bytes 2021-02-20T13:26:52.382 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:26:52.382 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:26:52.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:26:52.383 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:26:52.383 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:26:52.384 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:26:52.388 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:26:52.388 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T13:26:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:26:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:26:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:26:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:26:52.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_35 2021-02-20T13:26:52.536 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_35 snapshot 128M 2021-02-20T13:26:52.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:26:52.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:52.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:26:52.537 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_35 2021-02-20T13:26:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:26:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:26:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_35 128M 2021-02-20T13:26:52.541 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:26:52.541 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:52.542 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:26:52.543 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:52.543 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_35 2021-02-20T13:26:52.544 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:52.544 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:26:52.545 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:26:52.545 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:26:52.546 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:26:52.546 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_35 2021-02-20T13:26:52.614 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:26:52.614 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_35 snapshot 2021-02-20T13:26:52.614 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:26:52.615 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:26:52.615 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_35 2021-02-20T13:26:52.615 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:26:52.616 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_35 snapshot 2021-02-20T13:26:53.528 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:26:53.552 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:26:53.552 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_35 2021-02-20T13:26:53.552 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:26:53.553 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:26:53.553 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_35 2021-02-20T13:26:53.553 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:26:53.554 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=10 2021-02-20T13:26:53.554 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:26:53.554 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_35 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:26:54.267 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 69625856 bytes 2021-02-20T13:26:54.650 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 109916160 bytes 2021-02-20T13:26:54.877 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 119312384 bytes 2021-02-20T13:26:55.416 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 182337536 bytes 2021-02-20T13:26:55.743 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 233267200 bytes 2021-02-20T13:26:56.286 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 269865984 bytes 2021-02-20T13:26:56.621 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 294621184 bytes 2021-02-20T13:26:57.134 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 378016768 bytes 2021-02-20T13:26:57.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 385731584 bytes 2021-02-20T13:26:58.173 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 482167808 bytes 2021-02-20T13:26:58.581 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 509222912 bytes 2021-02-20T13:26:59.191 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 571910144 bytes 2021-02-20T13:26:59.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 605693952 bytes 2021-02-20T13:26:59.781 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 606306304 bytes 2021-02-20T13:27:00.196 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 646355968 bytes 2021-02-20T13:27:00.645 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 684044288 bytes 2021-02-20T13:27:01.275 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 715142144 bytes 2021-02-20T13:27:01.714 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 735017984 bytes 2021-02-20T13:27:02.407 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 838418432 bytes 2021-02-20T13:27:03.141 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 903467008 bytes 2021-02-20T13:27:03.444 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 920764416 bytes 2021-02-20T13:27:03.559 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:27:03.559 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:27:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:27:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:27:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_36 2021-02-20T13:27:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_36 snapshot 128M 2021-02-20T13:27:03.561 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:03.561 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:03.561 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:03.561 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_36 2021-02-20T13:27:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:27:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:27:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:03.563 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_36 128M 2021-02-20T13:27:03.563 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:03.563 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:03.563 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:03.564 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:03.564 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_36 2021-02-20T13:27:03.564 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:03.564 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:27:03.564 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:27:03.565 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:27:03.565 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:03.565 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_36 2021-02-20T13:27:03.627 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:27:03.627 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_36 snapshot 2021-02-20T13:27:03.628 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:03.628 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:03.628 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_36 2021-02-20T13:27:03.628 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:27:03.629 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_36 snapshot 2021-02-20T13:27:04.570 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:27:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:27:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_36 2021-02-20T13:27:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_36 2021-02-20T13:27:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:27:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=12 2021-02-20T13:27:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:27:04.594 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_36 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:27:04.869 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 615424 bytes 2021-02-20T13:27:05.113 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 8069120 bytes 2021-02-20T13:27:05.600 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 82400256 bytes 2021-02-20T13:27:05.978 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 136419328 bytes 2021-02-20T13:27:06.562 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 197547008 bytes 2021-02-20T13:27:06.788 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 199285760 bytes 2021-02-20T13:27:07.462 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 268205056 bytes 2021-02-20T13:27:08.016 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 329122816 bytes 2021-02-20T13:27:08.637 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 386944000 bytes 2021-02-20T13:27:09.074 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 410560512 bytes 2021-02-20T13:27:09.881 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 512802816 bytes 2021-02-20T13:27:10.290 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 533724160 bytes 2021-02-20T13:27:10.852 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 586441728 bytes 2021-02-20T13:27:11.451 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 646305792 bytes 2021-02-20T13:27:12.106 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 675980288 bytes 2021-02-20T13:27:12.601 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 736156672 bytes 2021-02-20T13:27:13.070 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 759535616 bytes 2021-02-20T13:27:13.327 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 769381376 bytes 2021-02-20T13:27:14.063 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 836728832 bytes 2021-02-20T13:27:14.616 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 895635456 bytes 2021-02-20T13:27:14.961 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 921163776 bytes 2021-02-20T13:27:15.541 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 979887104 bytes 2021-02-20T13:27:16.130 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 1036590080 bytes 2021-02-20T13:27:16.432 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 1048187904 bytes 2021-02-20T13:27:16.598 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:27:16.598 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:27:16.599 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:27:16.599 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:27:16.599 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_37 2021-02-20T13:27:16.599 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_37 snapshot 128M 2021-02-20T13:27:16.600 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:16.600 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:16.600 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:16.600 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:16.600 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_37 2021-02-20T13:27:16.601 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:16.601 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:27:16.601 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:27:16.601 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:16.601 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_37 128M 2021-02-20T13:27:16.602 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:16.602 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:16.602 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:16.603 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:16.603 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_37 2021-02-20T13:27:16.603 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:16.604 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:27:16.604 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:27:16.604 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:27:16.604 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:16.605 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_37 2021-02-20T13:27:16.676 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:27:16.677 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_37 snapshot 2021-02-20T13:27:16.677 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:16.677 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:16.678 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_37 2021-02-20T13:27:16.678 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:27:16.678 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_37 snapshot 2021-02-20T13:27:17.383 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:27:17.408 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:27:17.409 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_37 2021-02-20T13:27:17.409 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:17.409 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:17.409 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_37 2021-02-20T13:27:17.410 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:27:17.411 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=5 2021-02-20T13:27:17.411 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:27:17.411 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_37 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:27:17.981 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 38185984 bytes 2021-02-20T13:27:18.459 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 92299264 bytes 2021-02-20T13:27:19.069 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 145641472 bytes 2021-02-20T13:27:19.333 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 154438656 bytes 2021-02-20T13:27:19.887 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 227777536 bytes 2021-02-20T13:27:20.319 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 267766784 bytes 2021-02-20T13:27:20.939 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 349421568 bytes 2021-02-20T13:27:21.430 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 407346176 bytes 2021-02-20T13:27:21.703 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 411555840 bytes 2021-02-20T13:27:22.203 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 470394880 bytes 2021-02-20T13:27:22.420 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:27:22.420 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:27:22.421 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:27:22.421 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:27:22.421 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_38 2021-02-20T13:27:22.421 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_38 snapshot 128M 2021-02-20T13:27:22.422 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:22.422 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:22.422 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:22.422 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:22.422 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_38 2021-02-20T13:27:22.423 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:22.423 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:27:22.423 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:27:22.423 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:22.424 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_38 128M 2021-02-20T13:27:22.424 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:22.424 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:22.424 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:22.424 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:22.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_38 2021-02-20T13:27:22.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:22.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:27:22.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:27:22.426 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:27:22.426 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:22.426 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_38 2021-02-20T13:27:22.529 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:27:22.531 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_38 snapshot 2021-02-20T13:27:22.531 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:22.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:22.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_38 2021-02-20T13:27:22.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:27:22.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_38 snapshot 2021-02-20T13:27:22.735 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:27:22.763 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:27:22.763 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_38 2021-02-20T13:27:22.764 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:22.764 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:22.764 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_38 2021-02-20T13:27:22.764 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:27:22.765 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=5 2021-02-20T13:27:22.766 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:27:22.766 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_38 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:27:23.166 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 32833536 bytes 2021-02-20T13:27:23.739 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 119802880 bytes 2021-02-20T13:27:24.007 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 130228224 bytes 2021-02-20T13:27:25.691 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 254302208 bytes 2021-02-20T13:27:26.084 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 284289024 bytes 2021-02-20T13:27:26.365 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 301182976 bytes 2021-02-20T13:27:26.896 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 386466816 bytes 2021-02-20T13:27:27.387 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 428004352 bytes 2021-02-20T13:27:27.776 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:27:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:27:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:27:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:27:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_39 2021-02-20T13:27:27.778 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_39 snapshot 128M 2021-02-20T13:27:27.778 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:27.778 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:27.778 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:27.779 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:27.779 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_39 2021-02-20T13:27:27.779 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:27.779 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:27:27.779 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:27:27.780 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:27.780 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_39 128M 2021-02-20T13:27:27.780 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:27.780 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:27.780 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:27.781 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:27.781 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_39 2021-02-20T13:27:27.781 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:27.782 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:27:27.782 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:27:27.783 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:27:27.783 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:27.783 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_39 2021-02-20T13:27:27.865 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:27:27.865 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_39 snapshot 2021-02-20T13:27:27.865 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:27.866 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:27.866 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_39 2021-02-20T13:27:27.866 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:27:27.866 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_39 snapshot 2021-02-20T13:27:28.591 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:27:28.623 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:27:28.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_39 2021-02-20T13:27:28.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:28.625 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:28.625 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_39 2021-02-20T13:27:28.626 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:27:28.627 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=7 2021-02-20T13:27:28.627 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:27:29.186 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_39 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:27:29.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 5113856 bytes 2021-02-20T13:27:29.773 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 35365888 bytes 2021-02-20T13:27:30.324 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 118683648 bytes 2021-02-20T13:27:30.992 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 189597696 bytes 2021-02-20T13:27:31.503 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 251876352 bytes 2021-02-20T13:27:31.750 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 252518400 bytes 2021-02-20T13:27:32.091 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 299303936 bytes 2021-02-20T13:27:32.544 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 346786816 bytes 2021-02-20T13:27:32.805 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 352380928 bytes 2021-02-20T13:27:33.029 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 357319680 bytes 2021-02-20T13:27:33.305 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 366907392 bytes 2021-02-20T13:27:33.459 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6745012042567864) > weight (0.25) 2021-02-20T13:27:33.470 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5308044420698447) > weight (0.25) 2021-02-20T13:27:33.470 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.635119593696758) > weight (0.25) 2021-02-20T13:27:33.470 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9631032939949851) > weight (0.25) 2021-02-20T13:27:33.470 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T13:27:33.932 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 439309312 bytes 2021-02-20T13:27:34.223 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 453761024 bytes 2021-02-20T13:27:34.645 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 489851904 bytes 2021-02-20T13:27:35.333 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 563405824 bytes 2021-02-20T13:27:35.835 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 613283840 bytes 2021-02-20T13:27:36.192 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:27:36.193 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:27:36.193 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:27:36.194 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:27:36.194 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_40 2021-02-20T13:27:36.194 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_40 snapshot 128M 2021-02-20T13:27:36.195 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:36.195 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:36.196 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:36.196 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:36.196 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_40 2021-02-20T13:27:36.197 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:36.197 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:27:36.198 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:27:36.198 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:36.198 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_40 128M 2021-02-20T13:27:36.198 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:36.198 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:36.199 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:36.199 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:36.199 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_40 2021-02-20T13:27:36.199 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:36.199 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:27:36.200 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:27:36.200 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:27:36.200 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:36.200 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_40 2021-02-20T13:27:36.381 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:27:36.382 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_40 snapshot 2021-02-20T13:27:36.382 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:36.382 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:36.383 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_40 2021-02-20T13:27:36.383 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:27:36.383 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_40 snapshot 2021-02-20T13:27:37.498 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:27:37.522 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:27:37.523 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_40 2021-02-20T13:27:37.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:37.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:37.532 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_40 2021-02-20T13:27:37.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:27:37.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=8 2021-02-20T13:27:37.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:27:37.534 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_40 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:27:37.835 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 27228160 bytes 2021-02-20T13:27:38.379 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 97210368 bytes 2021-02-20T13:27:38.582 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 97820672 bytes 2021-02-20T13:27:38.903 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 124925952 bytes 2021-02-20T13:27:39.484 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 203104256 bytes 2021-02-20T13:27:40.063 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 244280320 bytes 2021-02-20T13:27:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 244881408 bytes 2021-02-20T13:27:40.946 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 343940096 bytes 2021-02-20T13:27:41.379 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 393112576 bytes 2021-02-20T13:27:41.726 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 423042048 bytes 2021-02-20T13:27:42.097 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 462371840 bytes 2021-02-20T13:27:42.408 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 483553280 bytes 2021-02-20T13:27:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 572688384 bytes 2021-02-20T13:27:43.577 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 645716992 bytes 2021-02-20T13:27:43.945 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 678577152 bytes 2021-02-20T13:27:44.350 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 720201728 bytes 2021-02-20T13:27:44.651 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 759299072 bytes 2021-02-20T13:27:45.405 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 848337920 bytes 2021-02-20T13:27:45.522 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:27:45.523 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:27:45.523 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:27:45.523 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:27:45.524 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_41 2021-02-20T13:27:45.524 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_41 snapshot 128M 2021-02-20T13:27:45.524 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:45.524 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:45.524 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:45.525 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:45.525 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_41 2021-02-20T13:27:45.525 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:45.526 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:27:45.526 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:27:45.526 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:45.526 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_41 128M 2021-02-20T13:27:45.527 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:45.527 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:45.527 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:45.527 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:45.527 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_41 2021-02-20T13:27:45.528 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:45.528 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:27:45.528 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:27:45.528 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:27:45.528 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:45.529 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_41 2021-02-20T13:27:45.587 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:27:45.588 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_41 snapshot 2021-02-20T13:27:45.588 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:45.588 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:45.588 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_41 2021-02-20T13:27:45.588 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:27:45.589 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_41 snapshot 2021-02-20T13:27:46.533 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:27:46.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:27:46.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_41 2021-02-20T13:27:46.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:46.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:46.558 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_41 2021-02-20T13:27:46.558 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:27:46.558 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=10 2021-02-20T13:27:46.559 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:27:46.559 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_41 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:27:46.843 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 5523456 bytes 2021-02-20T13:27:47.153 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 27638784 bytes 2021-02-20T13:27:47.600 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 78692352 bytes 2021-02-20T13:27:48.005 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 123341824 bytes 2021-02-20T13:27:48.463 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 178685952 bytes 2021-02-20T13:27:48.715 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 184782848 bytes 2021-02-20T13:27:48.978 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 199141376 bytes 2021-02-20T13:27:49.352 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 238459904 bytes 2021-02-20T13:27:49.710 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 263591936 bytes 2021-02-20T13:27:49.995 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 282804224 bytes 2021-02-20T13:27:50.624 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 329648128 bytes 2021-02-20T13:27:51.019 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 350821376 bytes 2021-02-20T13:27:51.509 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 398768128 bytes 2021-02-20T13:27:51.724 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 404710400 bytes 2021-02-20T13:27:52.196 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 448899072 bytes 2021-02-20T13:27:52.723 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 508027904 bytes 2021-02-20T13:27:53.369 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 591552512 bytes 2021-02-20T13:27:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 621218816 bytes 2021-02-20T13:27:54.018 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 661797888 bytes 2021-02-20T13:27:54.560 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 726757376 bytes 2021-02-20T13:27:55.274 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 784481280 bytes 2021-02-20T13:27:55.629 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 789799936 bytes 2021-02-20T13:27:56.034 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 823066624 bytes 2021-02-20T13:27:56.374 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 843786240 bytes 2021-02-20T13:27:56.564 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:27:56.564 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:27:56.565 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:27:56.565 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:27:56.565 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_42 2021-02-20T13:27:56.565 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_42 snapshot 128M 2021-02-20T13:27:56.566 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:56.566 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:56.566 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:56.566 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:56.566 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_42 2021-02-20T13:27:56.567 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:56.567 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:27:56.569 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:27:56.569 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:56.570 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_42 128M 2021-02-20T13:27:56.570 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:56.570 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:56.570 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:56.571 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:56.571 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_42 2021-02-20T13:27:56.571 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:56.571 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:27:56.572 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:27:56.572 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:27:56.572 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:27:56.572 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_42 2021-02-20T13:27:56.729 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:27:56.730 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_42 snapshot 2021-02-20T13:27:56.730 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:56.730 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:56.731 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_42 2021-02-20T13:27:56.731 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:27:56.731 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_42 snapshot 2021-02-20T13:27:57.384 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:27:57.389 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:27:57.390 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:27:57.391 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T13:27:57.391 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T13:27:57.388+0000 7f5d1554e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 31764) UID: 1000 2021-02-20T13:27:57.424 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:27:57.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_42 2021-02-20T13:27:57.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:27:57.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:27:57.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_42 2021-02-20T13:27:57.425 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:27:57.426 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=13 2021-02-20T13:27:57.426 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:27:57.427 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_42 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:27:57.709 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 633856 bytes 2021-02-20T13:27:57.794 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:27:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 25527296 bytes 2021-02-20T13:27:58.192 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 29995008 bytes 2021-02-20T13:27:58.535 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 63921152 bytes 2021-02-20T13:27:59.017 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 124894208 bytes 2021-02-20T13:27:59.473 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 175650816 bytes 2021-02-20T13:27:59.926 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 209805312 bytes 2021-02-20T13:28:00.544 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 289339392 bytes 2021-02-20T13:28:00.795 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 296549376 bytes 2021-02-20T13:28:01.175 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 332862464 bytes 2021-02-20T13:28:01.588 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 371392512 bytes 2021-02-20T13:28:01.869 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 385274880 bytes 2021-02-20T13:28:02.227 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 408307712 bytes 2021-02-20T13:28:02.392 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:28:02.392 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:28:02.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:28:02.393 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:28:02.393 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:28:02.394 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:28:02.396 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:28:02.396 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T13:28:02.472 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 415278080 bytes 2021-02-20T13:28:02.775 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 429946880 bytes 2021-02-20T13:28:03.364 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 493583360 bytes 2021-02-20T13:28:03.809 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 537262080 bytes 2021-02-20T13:28:04.230 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 576527360 bytes 2021-02-20T13:28:04.579 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 601185280 bytes 2021-02-20T13:28:05.003 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 634687488 bytes 2021-02-20T13:28:05.391 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 674802688 bytes 2021-02-20T13:28:06.242 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 741261312 bytes 2021-02-20T13:28:06.644 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 780739584 bytes 2021-02-20T13:28:07.217 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 835675136 bytes 2021-02-20T13:28:07.563 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 870679552 bytes 2021-02-20T13:28:08.038 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 928467968 bytes 2021-02-20T13:28:08.372 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 960204800 bytes 2021-02-20T13:28:08.626 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2800 IOs, 987400192 bytes 2021-02-20T13:28:08.866 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2900 IOs, 998595584 bytes 2021-02-20T13:28:09.281 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3000 IOs, 1058822144 bytes 2021-02-20T13:28:09.604 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3100 IOs, 1084098560 bytes 2021-02-20T13:28:10.034 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3200 IOs, 1141282816 bytes 2021-02-20T13:28:10.363 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3300 IOs, 1174415360 bytes 2021-02-20T13:28:10.430 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:28:10.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:28:10.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:28:10.431 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:28:10.432 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_43 2021-02-20T13:28:10.432 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_43 snapshot 128M 2021-02-20T13:28:10.432 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:28:10.432 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:10.432 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:28:10.433 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:10.433 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_43 2021-02-20T13:28:10.433 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:10.433 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:28:10.434 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:28:10.434 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:10.434 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_43 128M 2021-02-20T13:28:10.434 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:28:10.434 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:10.435 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:28:10.435 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:10.435 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_43 2021-02-20T13:28:10.435 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:10.436 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:28:10.436 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:28:10.436 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:28:10.436 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:10.436 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_43 2021-02-20T13:28:10.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:28:10.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_43 snapshot 2021-02-20T13:28:10.496 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:28:10.496 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:28:10.496 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_43 2021-02-20T13:28:10.496 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:28:10.497 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_43 snapshot 2021-02-20T13:28:11.378 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:28:11.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:28:11.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_43 2021-02-20T13:28:11.395 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:28:11.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:28:11.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_43 2021-02-20T13:28:11.396 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:28:11.397 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=11 2021-02-20T13:28:11.397 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:28:11.397 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_43 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:28:11.622 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 612352 bytes 2021-02-20T13:28:12.386 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 116844544 bytes 2021-02-20T13:28:12.775 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 140590080 bytes 2021-02-20T13:28:13.032 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 163935232 bytes 2021-02-20T13:28:13.285 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 186698752 bytes 2021-02-20T13:28:13.948 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 275412992 bytes 2021-02-20T13:28:14.243 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 307373056 bytes 2021-02-20T13:28:14.678 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 361120768 bytes 2021-02-20T13:28:15.073 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 408890368 bytes 2021-02-20T13:28:15.559 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 471375872 bytes 2021-02-20T13:28:15.880 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 471977984 bytes 2021-02-20T13:28:16.295 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 519078912 bytes 2021-02-20T13:28:16.556 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 535395328 bytes 2021-02-20T13:28:17.099 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 614493184 bytes 2021-02-20T13:28:17.393 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 621281280 bytes 2021-02-20T13:28:17.574 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 627340288 bytes 2021-02-20T13:28:17.925 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 665879552 bytes 2021-02-20T13:28:18.138 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 689717248 bytes 2021-02-20T13:28:18.690 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 748908544 bytes 2021-02-20T13:28:19.100 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 774009856 bytes 2021-02-20T13:28:19.391 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 802407424 bytes 2021-02-20T13:28:19.604 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 812246016 bytes 2021-02-20T13:28:20.005 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 862083072 bytes 2021-02-20T13:28:20.450 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 953877504 bytes 2021-02-20T13:28:21.346 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 1022770176 bytes 2021-02-20T13:28:21.589 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 1039847424 bytes 2021-02-20T13:28:22.054 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 1101051904 bytes 2021-02-20T13:28:22.403 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:28:22.403 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:28:22.403 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:28:22.404 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:28:22.404 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_44 2021-02-20T13:28:22.404 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_44 snapshot 128M 2021-02-20T13:28:22.404 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:28:22.405 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:22.405 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:28:22.405 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:22.405 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_44 2021-02-20T13:28:22.405 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:22.406 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:28:22.406 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:28:22.406 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:22.406 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_44 128M 2021-02-20T13:28:22.406 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:28:22.407 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:22.407 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:28:22.407 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:22.407 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_44 2021-02-20T13:28:22.408 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:22.408 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:28:22.408 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:28:22.408 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:28:22.409 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:22.409 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_44 2021-02-20T13:28:22.720 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:28:22.721 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_44 snapshot 2021-02-20T13:28:22.722 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:28:22.722 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:28:22.722 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_44 2021-02-20T13:28:22.723 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:28:22.723 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_44 snapshot 2021-02-20T13:28:22.906 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:28:22.926 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:28:22.927 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_44 2021-02-20T13:28:22.927 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:28:22.927 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:28:22.928 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_44 2021-02-20T13:28:22.928 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:28:22.929 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=14 2021-02-20T13:28:22.929 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:28:22.929 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_44 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:28:23.365 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 53590016 bytes 2021-02-20T13:28:23.676 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 89451520 bytes 2021-02-20T13:28:23.985 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 129081344 bytes 2021-02-20T13:28:24.449 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 190357504 bytes 2021-02-20T13:28:24.726 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 214905856 bytes 2021-02-20T13:28:25.214 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 279258112 bytes 2021-02-20T13:28:25.534 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 307359744 bytes 2021-02-20T13:28:25.822 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 358274048 bytes 2021-02-20T13:28:26.367 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 399366144 bytes 2021-02-20T13:28:26.525 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 399972352 bytes 2021-02-20T13:28:27.128 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 501978112 bytes 2021-02-20T13:28:27.455 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 532248576 bytes 2021-02-20T13:28:27.698 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 553584640 bytes 2021-02-20T13:28:28.019 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 587338752 bytes 2021-02-20T13:28:28.313 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 619325440 bytes 2021-02-20T13:28:28.653 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 680752128 bytes 2021-02-20T13:28:29.007 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 700558336 bytes 2021-02-20T13:28:29.368 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 741863424 bytes 2021-02-20T13:28:29.695 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 777123840 bytes 2021-02-20T13:28:30.169 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 851320832 bytes 2021-02-20T13:28:30.726 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 935215104 bytes 2021-02-20T13:28:31.245 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 994190336 bytes 2021-02-20T13:28:31.604 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 1052107776 bytes 2021-02-20T13:28:31.861 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 1059537920 bytes 2021-02-20T13:28:32.369 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 1132617728 bytes 2021-02-20T13:28:32.677 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 1167466496 bytes 2021-02-20T13:28:33.111 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 1225387008 bytes 2021-02-20T13:28:33.514 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2800 IOs, 1297258496 bytes 2021-02-20T13:28:34.078 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2900 IOs, 1356760064 bytes 2021-02-20T13:28:34.487 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3000 IOs, 1415812096 bytes 2021-02-20T13:28:34.891 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3100 IOs, 1488686080 bytes 2021-02-20T13:28:35.630 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3200 IOs, 1566380032 bytes 2021-02-20T13:28:36.098 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3300 IOs, 1589360640 bytes 2021-02-20T13:28:36.358 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3400 IOs, 1611163648 bytes 2021-02-20T13:28:36.663 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3500 IOs, 1639557120 bytes 2021-02-20T13:28:36.935 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:28:36.936 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:28:36.936 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:28:36.936 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:28:36.936 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_45 2021-02-20T13:28:36.937 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_45 snapshot 128M 2021-02-20T13:28:36.937 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:28:36.937 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:36.938 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:28:36.938 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:36.938 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_45 2021-02-20T13:28:36.938 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:36.938 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:28:36.939 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:28:36.939 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:36.939 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_45 128M 2021-02-20T13:28:36.939 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:28:36.939 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:36.940 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:28:36.940 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:36.940 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_45 2021-02-20T13:28:36.940 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:36.940 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:28:36.941 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:28:36.941 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:28:36.942 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:36.942 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_45 2021-02-20T13:28:37.063 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:28:37.063 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_45 snapshot 2021-02-20T13:28:37.063 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:28:37.064 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:28:37.064 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_45 2021-02-20T13:28:37.064 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:28:37.064 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_45 snapshot 2021-02-20T13:28:37.463 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:28:37.482 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:28:37.483 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_45 2021-02-20T13:28:37.483 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:28:37.483 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:28:37.483 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_45 2021-02-20T13:28:37.483 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:28:37.484 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=5 2021-02-20T13:28:37.484 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:28:37.485 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_45 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:28:37.949 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 64786432 bytes 2021-02-20T13:28:38.607 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 156343296 bytes 2021-02-20T13:28:38.971 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 187208704 bytes 2021-02-20T13:28:39.181 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 219154432 bytes 2021-02-20T13:28:39.707 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 266126336 bytes 2021-02-20T13:28:40.050 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 304184320 bytes 2021-02-20T13:28:40.230 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 306345984 bytes 2021-02-20T13:28:40.609 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 354900992 bytes 2021-02-20T13:28:40.931 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 385522688 bytes 2021-02-20T13:28:41.478 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 398294016 bytes 2021-02-20T13:28:42.138 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 519624704 bytes 2021-02-20T13:28:42.474 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 540568576 bytes 2021-02-20T13:28:42.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ error_code=124 2021-02-20T13:28:42.491 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set -e 2021-02-20T13:28:42.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ 124 -eq 124 ] 2021-02-20T13:28:42.492 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ return 0 2021-02-20T13:28:42.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ image=image_46 2021-02-20T13:28:42.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image_and_enable_mirror cluster2 mirror image_46 snapshot 128M 2021-02-20T13:28:42.493 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:28:42.494 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:42.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:28:42.495 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:42.496 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_46 2021-02-20T13:28:42.499 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:42.499 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:28:42.499 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n snapshot ] 2021-02-20T13:28:42.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:42.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ create_image cluster2 mirror image_46 128M 2021-02-20T13:28:42.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:28:42.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:42.500 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:28:42.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:42.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_46 2021-02-20T13:28:42.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:42.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local size=128 2021-02-20T13:28:42.501 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n 128M ] 2021-02-20T13:28:42.502 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ size=128M 2021-02-20T13:28:42.502 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ shift 2021-02-20T13:28:42.502 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_46 2021-02-20T13:28:42.555 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ image = image ] 2021-02-20T13:28:42.555 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ enable_mirror cluster2 mirror image_46 snapshot 2021-02-20T13:28:42.556 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:28:42.556 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:28:42.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_46 2021-02-20T13:28:42.557 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local mode=snapshot 2021-02-20T13:28:42.558 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_46 snapshot 2021-02-20T13:28:43.483 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:Mirroring enabled 2021-02-20T13:28:43.504 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ [ -n ] 2021-02-20T13:28:43.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ stress_write_image cluster2 mirror image_46 2021-02-20T13:28:43.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local cluster=cluster2 2021-02-20T13:28:43.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local pool=mirror 2021-02-20T13:28:43.505 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local image=image_46 2021-02-20T13:28:43.506 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T13:28:43.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ local duration=11 2021-02-20T13:28:43.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ set +e 2021-02-20T13:28:43.507 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_46 --debug-rbd=20 --debug-journaler=20 2021-02-20T13:28:43.985 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:100 IOs, 60664832 bytes 2021-02-20T13:28:44.231 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:200 IOs, 74519552 bytes 2021-02-20T13:28:44.449 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:300 IOs, 85990400 bytes 2021-02-20T13:28:44.714 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:400 IOs, 109829120 bytes 2021-02-20T13:28:45.031 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:500 IOs, 142842880 bytes 2021-02-20T13:28:45.217 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:600 IOs, 148045824 bytes 2021-02-20T13:28:45.488 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:700 IOs, 172867584 bytes 2021-02-20T13:28:45.682 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:800 IOs, 173500416 bytes 2021-02-20T13:28:45.942 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:900 IOs, 210333696 bytes 2021-02-20T13:28:46.554 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1000 IOs, 262707200 bytes 2021-02-20T13:28:47.376 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1100 IOs, 383541248 bytes 2021-02-20T13:28:47.730 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1200 IOs, 414883840 bytes 2021-02-20T13:28:47.995 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1300 IOs, 431226880 bytes 2021-02-20T13:28:48.161 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1400 IOs, 436858880 bytes 2021-02-20T13:28:48.403 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1500 IOs, 447830016 bytes 2021-02-20T13:28:48.633 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1600 IOs, 461408256 bytes 2021-02-20T13:28:48.990 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1700 IOs, 496978944 bytes 2021-02-20T13:28:49.458 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1800 IOs, 561852416 bytes 2021-02-20T13:28:49.802 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:1900 IOs, 606070784 bytes 2021-02-20T13:28:50.113 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2000 IOs, 628438016 bytes 2021-02-20T13:28:50.754 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2100 IOs, 709266432 bytes 2021-02-20T13:28:51.051 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2200 IOs, 763098112 bytes 2021-02-20T13:28:51.466 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2300 IOs, 785087488 bytes 2021-02-20T13:28:51.768 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2400 IOs, 788797440 bytes 2021-02-20T13:28:52.077 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2500 IOs, 833086464 bytes 2021-02-20T13:28:52.349 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2600 IOs, 844591104 bytes 2021-02-20T13:28:52.721 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2700 IOs, 889459712 bytes 2021-02-20T13:28:53.112 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2800 IOs, 945244160 bytes 2021-02-20T13:28:53.370 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:2900 IOs, 957859840 bytes 2021-02-20T13:28:53.854 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3000 IOs, 1022387200 bytes 2021-02-20T13:28:54.178 INFO:tasks.workunit.cluster1.client.mirror.smithi183.stdout:3100 IOs, 1054910464 bytes 2021-02-20T13:29:11.472 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4448238348981879) > weight (0.25) 2021-02-20T13:29:11.472 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9136239883361004) > weight (0.25) 2021-02-20T13:29:11.472 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7515259510229878) > weight (0.25) 2021-02-20T13:29:11.473 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5806471797503961) > weight (0.25) 2021-02-20T13:29:11.473 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T13:29:20.397 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6835947121268052) > weight (0.25) 2021-02-20T13:29:20.397 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5118531520588524) > weight (0.25) 2021-02-20T13:29:20.398 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.684742270839506) > weight (0.25) 2021-02-20T13:29:20.398 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T13:29:20.399 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T13:29:20.399 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T13:29:20.400 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T13:29:20.398+0000 7fb962298700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 25742) UID: 1000 2021-02-20T13:29:20.601 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:29:29.400 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T13:29:29.400 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:29:29.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:29:29.401 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T13:29:29.402 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T13:29:29.402 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T13:29:29.405 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T13:29:29.405 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T13:30:34.474 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7070908159830438) > weight (0.25) 2021-02-20T13:30:34.475 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T13:30:34.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T13:30:34.477 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T13:30:34.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:30:34.476+0000 7f6bafcaf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 30101) UID: 1000 2021-02-20T13:30:34.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:30:43.477 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T13:30:43.478 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:30:43.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:30:43.479 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T13:30:43.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T13:30:43.480 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T13:30:43.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T13:30:43.483 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T13:30:57.407 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:30:57.407 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:30:57.408 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T13:30:57.409 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T13:30:57.407+0000 7f70c4d5a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 32333) UID: 1000 2021-02-20T13:30:57.609 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:30:59.408 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:30:59.409 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:30:59.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:30:59.409 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:30:59.410 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:30:59.410 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:30:59.413 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:30:59.414 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T13:31:58.484 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:31:58.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:31:58.486 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T13:31:58.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T13:31:58.486+0000 7f95906c5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 30797) UID: 1000 2021-02-20T13:31:58.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:32:00.487 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:32:00.488 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:32:00.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:32:00.489 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:32:00.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:32:00.489 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:32:00.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:32:00.494 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T13:32:32.415 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.850176466846399) > weight (0.25) 2021-02-20T13:32:32.416 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5151749249727698) > weight (0.25) 2021-02-20T13:32:32.416 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5877631625306495) > weight (0.25) 2021-02-20T13:32:32.417 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T13:32:32.417 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T13:32:32.418 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T13:32:32.419 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T13:32:32.413+0000 7f25711e3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 32675) UID: 1000 2021-02-20T13:32:32.619 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:32:35.419 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T13:32:35.420 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:32:35.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:32:35.421 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T13:32:35.421 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T13:32:35.422 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T13:32:35.430 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T13:32:35.430 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T13:33:05.495 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.679696669867741) > weight (0.25) 2021-02-20T13:33:05.495 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3542429686544737) > weight (0.25) 2021-02-20T13:33:05.496 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.30767547351531366) > weight (0.25) 2021-02-20T13:33:05.496 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5721166084357238) > weight (0.25) 2021-02-20T13:33:05.496 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T13:34:26.431 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7144617552688999) > weight (0.25) 2021-02-20T13:34:26.432 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.45593549793084753) > weight (0.25) 2021-02-20T13:34:26.433 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T13:34:26.434 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T13:34:26.434 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T13:34:26.436 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T13:34:26.430+0000 7ffb1cbf0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 31160) UID: 1000 2021-02-20T13:34:26.635 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:34:29.435 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T13:34:29.436 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:34:29.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:34:29.438 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T13:34:29.438 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T13:34:29.439 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T13:34:29.450 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T13:34:29.450 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T13:34:34.498 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9997939316786314) > weight (0.25) 2021-02-20T13:34:34.498 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7893355038239563) > weight (0.25) 2021-02-20T13:34:34.498 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9976473333140583) > weight (0.25) 2021-02-20T13:34:34.499 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T13:34:34.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T13:34:34.500 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T13:34:34.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T13:34:34.498+0000 7f2095fa0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 19159) UID: 1000 2021-02-20T13:34:34.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:34:38.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T13:34:38.806+0000 7f5771e29700 -1 rbd::mirror::ImageReplayer: 0x55a2bd834300 [3/49a0b5eb-7d69-4824-806b-1ece12354eb9] start: already running 2021-02-20T13:34:39.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T13:34:39.410+0000 7f5771e29700 -1 rbd::mirror::ImageReplayer: 0x55a2c1726d00 [3/cdea041d-e1fe-4928-9842-3ce9553bc980] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:34:39.500 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T13:34:39.501 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:34:39.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:34:39.502 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T13:34:39.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T13:34:39.503 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T13:34:39.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T13:34:39.505 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T13:35:35.452 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9774856521409345) > weight (0.25) 2021-02-20T13:35:35.453 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7276989350947521) > weight (0.25) 2021-02-20T13:35:35.453 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.923283764301965) > weight (0.25) 2021-02-20T13:35:35.453 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5343495875325677) > weight (0.25) 2021-02-20T13:35:35.453 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T13:36:04.506 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5589662468375467) > weight (0.25) 2021-02-20T13:36:04.507 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T13:36:04.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T13:36:04.509 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T13:36:04.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:36:04.508+0000 7efef6229700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 32767) UID: 1000 2021-02-20T13:36:04.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:36:07.509 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T13:36:07.510 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:36:07.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:36:07.511 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T13:36:07.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T13:36:07.511 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T13:36:07.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T13:36:07.515 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T13:36:58.455 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:36:58.456 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:36:58.457 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T13:36:58.457 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T13:36:58.454+0000 7f5ac6293700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 32858) UID: 1000 2021-02-20T13:36:58.658 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:37:00.458 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:37:00.458 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:37:00.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:37:00.459 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:37:00.459 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:37:00.460 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:37:00.463 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:37:00.463 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T13:37:21.516 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9936203136275973) > weight (0.25) 2021-02-20T13:37:21.516 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6775609890836153) > weight (0.25) 2021-02-20T13:37:21.517 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T13:37:21.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T13:37:21.517 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T13:37:21.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T13:37:21.514+0000 7f8a1cf0d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 31612) UID: 1000 2021-02-20T13:37:21.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:37:25.519 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T13:37:25.519 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:37:25.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:37:25.520 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T13:37:25.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T13:37:25.521 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T13:37:25.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T13:37:25.524 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T13:38:30.464 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6895082017810669) > weight (0.25) 2021-02-20T13:38:30.465 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.638126505354491) > weight (0.25) 2021-02-20T13:38:30.465 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.46906652819954353) > weight (0.25) 2021-02-20T13:38:30.465 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5143515514424807) > weight (0.25) 2021-02-20T13:38:30.466 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T13:38:34.525 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:38:34.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:38:34.528 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T13:38:34.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T13:38:34.528+0000 7f5772e2b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 32949) UID: 1000 2021-02-20T13:38:34.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:38:38.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x558875547b80 [3/172baefb-5ce5-45e0-997e-249560de6fbd] start: already running 2021-02-20T13:38:38.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x558875547e00 [3/1a170226-518b-4a75-85a8-6fbdc9b5d9f0] start: already running 2021-02-20T13:38:38.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x558875548080 [3/205dce0d-bc7c-41b6-8ed9-87e6a63d1a70] start: already running 2021-02-20T13:38:38.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x558875548300 [3/2464eeeb-6954-426d-a8e5-129943cea55c] start: already running 2021-02-20T13:38:38.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x558875548580 [3/3ab5666f-763c-442d-bc17-ea037dcc3a42] start: already running 2021-02-20T13:38:38.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x558875548800 [3/3c6940ee-a642-422e-aafb-e3920bba87cf] start: already running 2021-02-20T13:38:38.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x558875548a80 [3/3d38caf8-6420-4f62-95dc-628caa5144da] start: already running 2021-02-20T13:38:38.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x558875548d00 [3/49a0b5eb-7d69-4824-806b-1ece12354eb9] start: already running 2021-02-20T13:38:38.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x558875548f80 [3/4c862af6-d1da-4215-b6b8-461f3883b104] start: already running 2021-02-20T13:38:38.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x558875549200 [3/4cba354e-99d2-4859-a4d0-baeec2203656] start: already running 2021-02-20T13:38:38.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x558875549480 [3/567ace7d-44e4-4d42-b47a-38bd35ec0e11] start: already running 2021-02-20T13:38:38.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x558875549700 [3/5786ec48-0da5-470d-b6a6-ddd71a666657] start: already running 2021-02-20T13:38:38.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x558875549980 [3/5c036645-b242-4fa8-bde0-d5b28dda3fd3] start: already running 2021-02-20T13:38:38.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x558875549c00 [3/5f3204b5-90c6-4244-a940-e749860457d5] start: already running 2021-02-20T13:38:38.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794ae000 [3/675a1b7b-4e17-4b1e-9e92-eaf406f1c33f] start: already running 2021-02-20T13:38:38.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794ae280 [3/681dbea5-7557-4dba-a92d-01da85613e31] start: already running 2021-02-20T13:38:38.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794ae500 [3/753c14e7-97d4-4109-8814-2c7f1abbbee8] start: already running 2021-02-20T13:38:38.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794ae780 [3/7658e9e0-5116-4c47-bad5-4e7894e27da7] start: already running 2021-02-20T13:38:38.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x558875547b80 [3/172baefb-5ce5-45e0-997e-249560de6fbd] start: already running 2021-02-20T13:38:38.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x558875547e00 [3/1a170226-518b-4a75-85a8-6fbdc9b5d9f0] start: already running 2021-02-20T13:38:38.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x558875548080 [3/205dce0d-bc7c-41b6-8ed9-87e6a63d1a70] start: already running 2021-02-20T13:38:38.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x558875548300 [3/2464eeeb-6954-426d-a8e5-129943cea55c] start: already running 2021-02-20T13:38:38.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x558875548580 [3/3ab5666f-763c-442d-bc17-ea037dcc3a42] start: already running 2021-02-20T13:38:38.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x558875548800 [3/3c6940ee-a642-422e-aafb-e3920bba87cf] start: already running 2021-02-20T13:38:38.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x558875548a80 [3/3d38caf8-6420-4f62-95dc-628caa5144da] start: already running 2021-02-20T13:38:38.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x558875548d00 [3/49a0b5eb-7d69-4824-806b-1ece12354eb9] start: already running 2021-02-20T13:38:38.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x558875548f80 [3/4c862af6-d1da-4215-b6b8-461f3883b104] start: already running 2021-02-20T13:38:38.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x558875549200 [3/4cba354e-99d2-4859-a4d0-baeec2203656] start: already running 2021-02-20T13:38:38.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x558875549480 [3/567ace7d-44e4-4d42-b47a-38bd35ec0e11] start: already running 2021-02-20T13:38:38.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x558875549700 [3/5786ec48-0da5-470d-b6a6-ddd71a666657] start: already running 2021-02-20T13:38:38.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x558875549980 [3/5c036645-b242-4fa8-bde0-d5b28dda3fd3] start: already running 2021-02-20T13:38:38.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x558875549c00 [3/5f3204b5-90c6-4244-a940-e749860457d5] start: already running 2021-02-20T13:38:38.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794ae000 [3/675a1b7b-4e17-4b1e-9e92-eaf406f1c33f] start: already running 2021-02-20T13:38:38.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588794ae280 [3/681dbea5-7557-4dba-a92d-01da85613e31] start: already running 2021-02-20T13:38:38.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588794ae500 [3/753c14e7-97d4-4109-8814-2c7f1abbbee8] start: already running 2021-02-20T13:38:38.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.856+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794ae780 [3/7658e9e0-5116-4c47-bad5-4e7894e27da7] start: already running 2021-02-20T13:38:38.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.860+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588794aea00 [3/790c038c-4353-4ff1-ae0b-9c9bb1e1adaa] start: already running 2021-02-20T13:38:38.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.860+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588794aec80 [3/7fb27511-6f76-444f-b338-191b2c7caa61] start: already running 2021-02-20T13:38:38.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.860+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588794aef00 [3/858c7e91-92b9-4e8b-8ae9-f8cb46e242bf] start: already running 2021-02-20T13:38:38.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.860+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588794af180 [3/87eed4f8-1d97-46e0-b439-8f6b85b99b40] start: already running 2021-02-20T13:38:38.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.860+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588794b0080 [3/8c216315-5117-4846-b850-c4265b4697a3] start: already running 2021-02-20T13:38:38.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.860+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588794b0300 [3/91672d77-7c82-4d00-b945-306681ce602c] start: already running 2021-02-20T13:38:38.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.860+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588794b0580 [3/941323f6-0453-4944-ab79-664fde6ec580] start: already running 2021-02-20T13:38:38.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.860+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588794b0800 [3/945695ad-0f15-4fd7-a092-4e7df53e1d1b] start: already running 2021-02-20T13:38:38.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.860+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588794b0a80 [3/972a582c-01df-4309-8926-a24b908ec0fc] start: already running 2021-02-20T13:38:38.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.860+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794af400 [3/9940675f-5679-4d59-9726-da9895d6bb7c] start: already running 2021-02-20T13:38:38.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.860+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794af680 [3/9b42b7b2-0767-46b8-a6a6-e0b14c0863eb] start: already running 2021-02-20T13:38:38.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.860+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794b0d00 [3/9bb53f74-14d3-4057-8f64-9bba63f946e2] start: already running 2021-02-20T13:38:38.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.860+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794b0f80 [3/a3cf9f38-0520-4671-b928-6d907b15b2fb] start: already running 2021-02-20T13:38:38.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.860+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794af900 [3/a3d3aab0-3900-425e-a825-80fa9ff092ea] start: already running 2021-02-20T13:38:38.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.860+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794afb80 [3/ab28f848-782d-4c9d-a26a-5bdb09404113] start: already running 2021-02-20T13:38:38.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.860+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794afe00 [3/b1b702c0-1f24-4c9b-b9bc-cc3a0752ca38] start: already running 2021-02-20T13:38:38.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.860+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794b1980 [3/b97c3e7d-cef7-4890-a315-ffa1f7e51804] start: already running 2021-02-20T13:38:38.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:38.860+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794b1c00 [3/bbf1cffa-5e93-4c1c-937d-71c8c54806ff] start: already running 2021-02-20T13:38:39.151 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T13:38:39.144+0000 7f03ccd38700 -1 reset not still connected to 0x556b850be410 2021-02-20T13:38:39.152 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T13:38:39.144+0000 7f03ccd38700 -1 reset not still connected to 0x556b85c895a0 2021-02-20T13:38:39.152 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T13:38:39.144+0000 7f03ccd38700 -1 reset not still connected to 0x556b8794dba0 2021-02-20T13:38:39.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.244+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794b1980 [3/b97c3e7d-cef7-4890-a315-ffa1f7e51804] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.244+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588794b1980 [3/b97c3e7d-cef7-4890-a315-ffa1f7e51804] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.244+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794ae000 [3/675a1b7b-4e17-4b1e-9e92-eaf406f1c33f] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.244+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588794ae000 [3/675a1b7b-4e17-4b1e-9e92-eaf406f1c33f] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.244+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794b1480 [3/cdfd4b53-cfc9-488d-957e-07aa3d636acf] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.244+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588794b1480 [3/cdfd4b53-cfc9-488d-957e-07aa3d636acf] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.244+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794b0a80 [3/972a582c-01df-4309-8926-a24b908ec0fc] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.244+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588794b0a80 [3/972a582c-01df-4309-8926-a24b908ec0fc] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.284+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588794ae780 [3/7658e9e0-5116-4c47-bad5-4e7894e27da7] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.284+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794ae780 [3/7658e9e0-5116-4c47-bad5-4e7894e27da7] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.284+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588794b1200 [3/cdea041d-e1fe-4928-9842-3ce9553bc980] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.284+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794b1200 [3/cdea041d-e1fe-4928-9842-3ce9553bc980] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.284+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588794ae500 [3/753c14e7-97d4-4109-8814-2c7f1abbbee8] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.284+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794ae500 [3/753c14e7-97d4-4109-8814-2c7f1abbbee8] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.284+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588794afe00 [3/b1b702c0-1f24-4c9b-b9bc-cc3a0752ca38] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.284+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794afe00 [3/b1b702c0-1f24-4c9b-b9bc-cc3a0752ca38] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.284+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588794b0d00 [3/9bb53f74-14d3-4057-8f64-9bba63f946e2] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.284+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794b0d00 [3/9bb53f74-14d3-4057-8f64-9bba63f946e2] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.284+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x558875548580 [3/3ab5666f-763c-442d-bc17-ea037dcc3a42] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.284+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x558875548580 [3/3ab5666f-763c-442d-bc17-ea037dcc3a42] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.284+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588794aec80 [3/7fb27511-6f76-444f-b338-191b2c7caa61] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.284+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588794aec80 [3/7fb27511-6f76-444f-b338-191b2c7caa61] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.284+0000 7fac08150700 -1 rbd::mirror::ImageReplayer: 0x5588795caf00 [3/ff38d3a1-22f6-4a7c-a814-2d4505cf6570] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:39.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:38:39.284+0000 7fac0794f700 -1 rbd::mirror::ImageReplayer: 0x5588795caf00 [3/ff38d3a1-22f6-4a7c-a814-2d4505cf6570] register_admin_socket_hook: error registering admin socket commands 2021-02-20T13:38:41.529 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:38:41.529 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:38:41.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:38:41.530 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:38:41.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:38:41.531 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:38:41.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:38:41.534 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T13:39:49.535 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6358471131174409) > weight (0.25) 2021-02-20T13:39:49.536 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8044837998118307) > weight (0.25) 2021-02-20T13:39:49.536 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T13:39:49.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T13:39:49.537 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T13:39:49.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T13:39:49.534+0000 7fa0c6272700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 33510) UID: 1000 2021-02-20T13:39:49.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:39:57.539 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T13:39:57.539 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:39:57.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:39:57.540 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T13:39:57.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T13:39:57.541 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T13:39:57.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T13:39:57.544 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T13:40:02.467 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7007819952791376) > weight (0.25) 2021-02-20T13:40:02.468 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7419004680625015) > weight (0.25) 2021-02-20T13:40:02.468 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8017827679370554) > weight (0.25) 2021-02-20T13:40:02.469 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7971697386217813) > weight (0.25) 2021-02-20T13:40:02.469 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T13:41:43.545 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7735708877089632) > weight (0.25) 2021-02-20T13:41:43.546 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6508154923811265) > weight (0.25) 2021-02-20T13:41:43.546 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T13:41:43.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T13:41:43.547 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T13:41:43.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T13:41:43.547+0000 7ff68c946700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 33702) UID: 1000 2021-02-20T13:41:43.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:41:45.548 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T13:41:45.549 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:41:45.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:41:45.550 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T13:41:45.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T13:41:45.550 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T13:41:45.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T13:41:45.553 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T13:41:46.470 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5200163997006627) > weight (0.25) 2021-02-20T13:41:46.470 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7893951662295537) > weight (0.25) 2021-02-20T13:41:46.470 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5340999838190937) > weight (0.25) 2021-02-20T13:41:46.471 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5896043399741534) > weight (0.25) 2021-02-20T13:41:46.471 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T13:42:49.472 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.623431594337689) > weight (0.25) 2021-02-20T13:42:49.473 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.859841115696399) > weight (0.25) 2021-02-20T13:42:49.473 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8848128471605007) > weight (0.25) 2021-02-20T13:42:49.473 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7792175982873439) > weight (0.25) 2021-02-20T13:42:49.474 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T13:43:44.555 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:43:44.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:43:44.557 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T13:43:44.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T13:43:44.556+0000 7fd27d43e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 33609) UID: 1000 2021-02-20T13:43:44.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:43:48.558 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:43:48.558 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:43:48.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:43:48.559 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:43:48.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:43:48.560 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:43:48.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:43:48.563 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T13:44:21.475 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:44:21.478 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:44:21.479 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T13:44:21.479 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T13:44:21.479+0000 7f4d89d1a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 33419) UID: 1000 2021-02-20T13:44:21.680 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:44:25.480 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:44:25.480 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:44:25.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:44:25.481 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:44:25.481 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:44:25.482 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:44:25.485 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:44:25.485 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T13:45:33.564 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5153319807465576) > weight (0.25) 2021-02-20T13:45:33.564 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3054079615118994) > weight (0.25) 2021-02-20T13:45:33.565 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6828507845134778) > weight (0.25) 2021-02-20T13:45:33.565 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T13:45:33.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T13:45:33.566 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T13:45:33.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T13:45:33.565+0000 7f3a4ed73700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 33232) UID: 1000 2021-02-20T13:45:33.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:45:38.567 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T13:45:38.568 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:45:38.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:45:38.569 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T13:45:38.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T13:45:38.570 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T13:45:38.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T13:45:38.572 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T13:46:24.488 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4189381252992296) > weight (0.25) 2021-02-20T13:46:24.489 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8160656199280606) > weight (0.25) 2021-02-20T13:46:24.489 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5503691827746303) > weight (0.25) 2021-02-20T13:46:24.489 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T13:46:24.492 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T13:46:24.492 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T13:46:24.493 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T13:46:24.492+0000 7f02949f1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 33040) UID: 1000 2021-02-20T13:46:24.693 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:46:30.494 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T13:46:30.494 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:46:30.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:46:30.495 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T13:46:30.495 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T13:46:30.496 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T13:46:30.499 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T13:46:30.499 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T13:46:55.574 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4741324517843051) > weight (0.25) 2021-02-20T13:46:55.680 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6663359808307245) > weight (0.25) 2021-02-20T13:46:55.681 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6831840983805045) > weight (0.25) 2021-02-20T13:46:55.681 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7613429892598299) > weight (0.25) 2021-02-20T13:46:55.681 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T13:48:02.500 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5310686000502558) > weight (0.25) 2021-02-20T13:48:02.501 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.27567816586555316) > weight (0.25) 2021-02-20T13:48:02.501 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5630328646096864) > weight (0.25) 2021-02-20T13:48:02.502 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.25946127768576566) > weight (0.25) 2021-02-20T13:48:02.502 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T13:48:26.682 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:48:26.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:48:26.684 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T13:48:26.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T13:48:26.684+0000 7f30b4339700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 33884) UID: 1000 2021-02-20T13:48:26.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:48:34.686 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:48:34.686 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:48:34.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:48:34.687 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:48:34.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:48:34.688 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:48:34.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:48:34.692 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T13:49:18.503 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:49:18.505 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:49:18.505 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T13:49:18.506 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T13:49:18.503+0000 7ff424a75700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 33977) UID: 1000 2021-02-20T13:49:18.706 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:49:20.506 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:49:20.506 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:49:20.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:49:20.507 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:49:20.507 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:49:20.508 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:49:20.511 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:49:20.511 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T13:49:47.693 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5580462240999526) > weight (0.25) 2021-02-20T13:49:47.694 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4487809530920678) > weight (0.25) 2021-02-20T13:49:47.694 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9209060078132975) > weight (0.25) 2021-02-20T13:49:47.694 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T13:49:47.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T13:49:47.695 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T13:49:47.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T13:49:47.694+0000 7ff66d4ca700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 34072) UID: 1000 2021-02-20T13:49:47.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:49:54.697 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T13:49:54.697 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:49:54.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:49:54.698 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T13:49:54.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T13:49:54.699 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T13:49:54.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T13:49:54.702 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T13:51:02.512 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:51:02.514 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:51:02.514 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T13:51:02.515 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:51:02.515 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:51:02.518 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T13:51:02.516+0000 7fa1aab77700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 34345) UID: 1000 2021-02-20T13:51:02.719 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:51:08.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:51:08.618 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:51:08.619 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:51:08.620 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:51:08.626 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:51:08.627 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T13:51:23.703 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6257770094384492) > weight (0.25) 2021-02-20T13:51:23.703 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T13:51:23.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T13:51:23.704 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T13:51:23.705 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T13:51:23.705 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:51:23.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:51:23.704+0000 7fac08951700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 33327) UID: 1000 2021-02-20T13:51:24.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:51:29.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:51:29.807 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T13:51:29.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T13:51:29.808 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T13:51:29.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T13:51:29.811 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T13:52:36.812 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.30705247992077844) > weight (0.25) 2021-02-20T13:52:36.813 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9868391548099326) > weight (0.25) 2021-02-20T13:52:36.813 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6667773633839914) > weight (0.25) 2021-02-20T13:52:36.813 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T13:52:36.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T13:52:36.814 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T13:52:36.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T13:52:36.814+0000 7f9c7f671700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 34438) UID: 1000 2021-02-20T13:52:37.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:52:45.816 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T13:52:45.817 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:52:45.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:52:45.818 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T13:52:45.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T13:52:45.819 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T13:52:45.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T13:52:45.824 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T13:52:53.628 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.578072291300943) > weight (0.25) 2021-02-20T13:52:53.628 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4486174343035533) > weight (0.25) 2021-02-20T13:52:53.629 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8162249024488314) > weight (0.25) 2021-02-20T13:52:53.629 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4917817431300634) > weight (0.25) 2021-02-20T13:52:53.629 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T13:53:48.826 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.28989861295468844) > weight (0.25) 2021-02-20T13:53:48.826 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6294287295121417) > weight (0.25) 2021-02-20T13:53:48.827 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T13:53:48.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T13:53:48.828 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T13:53:48.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T13:53:48.828+0000 7f681a0f8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 33793) UID: 1000 2021-02-20T13:53:49.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:53:56.829 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T13:53:56.829 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:53:56.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:53:56.830 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T13:53:56.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T13:53:56.831 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T13:53:56.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T13:53:56.834 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T13:54:12.630 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6636345009091436) > weight (0.25) 2021-02-20T13:54:12.630 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:54:12.631 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:54:12.631 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T13:54:12.632 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T13:54:12.631+0000 7f6fe07ba700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 30453) UID: 1000 2021-02-20T13:54:12.833 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:54:13.632 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T13:54:13.633 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:54:13.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:54:13.634 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T13:54:13.634 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:54:13.635 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:54:13.637 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:54:13.638 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T13:54:16.844 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T13:54:16.843+0000 7f9654912700 -1 rbd::mirror::ImageReplayer: 0x55a8a0731e00 [3/205dce0d-bc7c-41b6-8ed9-87e6a63d1a70] start: already running 2021-02-20T13:54:16.845 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T13:54:16.843+0000 7f9654912700 -1 rbd::mirror::ImageReplayer: 0x55a8a0732080 [3/2464eeeb-6954-426d-a8e5-129943cea55c] start: already running 2021-02-20T13:54:16.845 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T13:54:16.843+0000 7f9654912700 -1 rbd::mirror::ImageReplayer: 0x55a8a0732300 [3/3ab5666f-763c-442d-bc17-ea037dcc3a42] start: already running 2021-02-20T13:54:16.845 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T13:54:16.843+0000 7f9654912700 -1 rbd::mirror::ImageReplayer: 0x55a8a0732580 [3/3c6940ee-a642-422e-aafb-e3920bba87cf] start: already running 2021-02-20T13:54:16.846 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T13:54:16.843+0000 7f9654111700 -1 rbd::mirror::ImageReplayer: 0x55a8a47ec780 [3/eafa574f-c2c8-4659-b559-be75ec7d56bb] start: already running 2021-02-20T13:54:16.847 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T13:54:16.843+0000 7f9654111700 -1 rbd::mirror::ImageReplayer: 0x55a8a47eca00 [3/ef164021-65e4-434b-a5d7-bcdd53df22fd] start: already running 2021-02-20T13:54:16.847 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T13:54:16.843+0000 7f9654111700 -1 rbd::mirror::ImageReplayer: 0x55a8a47ecc80 [3/ff38d3a1-22f6-4a7c-a814-2d4505cf6570] start: already running 2021-02-20T13:54:16.847 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T13:54:16.847+0000 7f9654111700 -1 rbd::mirror::ImageReplayer: 0x55a8a47ecf00 [3/ff64bb69-23c4-4b3a-9b65-a48c78854429] start: already running 2021-02-20T13:54:59.835 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4354319161805942) > weight (0.25) 2021-02-20T13:54:59.860 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.395383616927536) > weight (0.25) 2021-02-20T13:54:59.860 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T13:54:59.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T13:54:59.862 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T13:54:59.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T13:54:59.858+0000 7f01eeefa700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 34820) UID: 1000 2021-02-20T13:55:00.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:55:06.863 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T13:55:06.891 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:55:06.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:55:06.892 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T13:55:06.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T13:55:06.892 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T13:55:06.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T13:55:06.895 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T13:55:42.638 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4138097986131195) > weight (0.25) 2021-02-20T13:55:42.639 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7943920359657667) > weight (0.25) 2021-02-20T13:55:42.639 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T13:55:42.640 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T13:55:42.640 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T13:55:42.641 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T13:55:42.641+0000 7f50b83a4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 33133) UID: 1000 2021-02-20T13:55:42.841 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:55:49.641 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T13:55:49.642 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:55:49.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:55:49.643 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T13:55:49.643 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T13:55:49.643 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T13:55:49.646 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T13:55:49.646 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T13:56:06.897 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8796241284655308) > weight (0.25) 2021-02-20T13:56:06.898 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T13:56:06.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T13:56:06.899 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T13:56:06.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T13:56:06.900+0000 7f1284116700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 34630) UID: 1000 2021-02-20T13:56:07.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:56:07.901 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T13:56:07.901 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:56:07.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:56:07.902 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T13:56:07.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T13:56:07.902 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T13:56:07.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T13:56:07.906 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T13:57:15.648 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4359408404135614) > weight (0.25) 2021-02-20T13:57:15.649 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6521294064811475) > weight (0.25) 2021-02-20T13:57:15.649 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T13:57:15.650 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T13:57:15.650 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T13:57:15.651 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T13:57:15.650+0000 7f8767233700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 35099) UID: 1000 2021-02-20T13:57:15.851 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:57:21.650 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T13:57:21.651 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:57:21.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:57:21.652 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T13:57:21.652 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T13:57:21.653 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T13:57:21.656 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T13:57:21.656 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T13:57:30.774 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T13:57:30.770+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a5679400 2021-02-20T13:57:30.774 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T13:57:30.770+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b9ff5ad0 2021-02-20T13:58:02.906 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:58:02.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:58:02.908 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T13:58:02.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T13:58:02.905+0000 7f5c461c0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 34254) UID: 1000 2021-02-20T13:58:03.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:58:05.909 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:58:05.914 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:58:05.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:58:05.915 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:58:05.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:58:05.916 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:58:05.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:58:05.919 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T13:58:26.657 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.976857078725983) > weight (0.25) 2021-02-20T13:58:26.665 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.40702312131165086) > weight (0.25) 2021-02-20T13:58:26.666 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2963095779566297) > weight (0.25) 2021-02-20T13:58:26.666 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2924324587157727) > weight (0.25) 2021-02-20T13:58:26.666 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T13:59:45.919 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:59:45.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:59:45.921 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T13:59:45.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T13:59:45.918+0000 7f884ec60700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 35372) UID: 1000 2021-02-20T13:59:46.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:59:51.922 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:59:51.923 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:59:51.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:59:51.923 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:59:51.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:59:51.924 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:59:51.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:59:51.927 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T14:00:10.668 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3353532233267482) > weight (0.25) 2021-02-20T14:00:10.668 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.65885469692416) > weight (0.25) 2021-02-20T14:00:10.669 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5926741483279344) > weight (0.25) 2021-02-20T14:00:10.669 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T14:00:10.670 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T14:00:10.671 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T14:00:10.672 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T14:00:10.670+0000 7f9655113700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 34163) UID: 1000 2021-02-20T14:00:10.877 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:00:19.672 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T14:00:19.673 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:00:19.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:00:19.674 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T14:00:19.674 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T14:00:19.675 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T14:00:19.678 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T14:00:19.679 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T14:01:50.929 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7692764406626215) > weight (0.25) 2021-02-20T14:01:50.929 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9091204135934767) > weight (0.25) 2021-02-20T14:01:50.930 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9360500956469324) > weight (0.25) 2021-02-20T14:01:50.930 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8111508680322743) > weight (0.25) 2021-02-20T14:01:50.930 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T14:02:11.680 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7026992584406998) > weight (0.25) 2021-02-20T14:02:11.690 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:02:11.692 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:02:11.692 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T14:02:11.693 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T14:02:11.690+0000 7fbeafec2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 34911) UID: 1000 2021-02-20T14:02:11.894 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:02:19.693 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:02:19.707 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:02:19.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:02:19.709 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:02:19.709 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:02:19.710 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:02:19.716 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:02:19.716 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T14:03:36.718 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5053435378701926) > weight (0.25) 2021-02-20T14:03:36.728 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.625749178703776) > weight (0.25) 2021-02-20T14:03:36.728 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T14:03:36.729 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T14:03:36.730 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T14:03:36.731 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T14:03:36.728+0000 7fdfed732700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 35281) UID: 1000 2021-02-20T14:03:36.931 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:03:37.931 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7958841844530691) > weight (0.25) 2021-02-20T14:03:37.932 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T14:03:37.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T14:03:37.932 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T14:03:37.933 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T14:03:37.933 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:03:37.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T14:03:37.932+0000 7f47a9c57700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 35190) UID: 1000 2021-02-20T14:03:38.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:03:39.730 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T14:03:39.752 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:03:39.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:03:39.753 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T14:03:39.753 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T14:03:39.754 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T14:03:39.757 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T14:03:39.757 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T14:03:44.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:03:44.035 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T14:03:44.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T14:03:44.035 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T14:03:44.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T14:03:44.038 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T14:04:46.040 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.44818433899523213) > weight (0.25) 2021-02-20T14:04:46.049 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T14:04:46.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T14:04:46.050 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T14:04:46.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T14:04:46.050+0000 7f71f0238700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 35907) UID: 1000 2021-02-20T14:04:46.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:04:54.051 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T14:04:54.058 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:04:54.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:04:54.060 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T14:04:54.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T14:04:54.061 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T14:04:54.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T14:04:54.066 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T14:05:00.758 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8708069960005517) > weight (0.25) 2021-02-20T14:05:00.759 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6162538509813491) > weight (0.25) 2021-02-20T14:05:00.759 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7211144986338062) > weight (0.25) 2021-02-20T14:05:00.759 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.40558357403454737) > weight (0.25) 2021-02-20T14:05:00.760 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T14:06:36.068 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3944669482820756) > weight (0.25) 2021-02-20T14:06:36.075 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8787385057386108) > weight (0.25) 2021-02-20T14:06:36.076 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8283517778578525) > weight (0.25) 2021-02-20T14:06:36.076 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T14:06:36.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T14:06:36.077 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T14:06:36.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T14:06:36.075+0000 7fce51c31700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 34721) UID: 1000 2021-02-20T14:06:36.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:06:40.761 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:06:40.761 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:06:40.762 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T14:06:40.763 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T14:06:40.763+0000 7f1216ca3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 34531) UID: 1000 2021-02-20T14:06:40.963 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:06:44.078 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T14:06:44.078 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:06:44.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:06:44.079 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T14:06:44.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T14:06:44.080 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T14:06:44.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T14:06:44.083 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T14:06:47.763 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:06:47.763 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:06:47.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:06:47.765 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:06:47.765 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:06:47.766 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:06:47.768 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:06:47.769 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T14:06:54.356 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T14:06:54.355+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a391a820 2021-02-20T14:06:54.358 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T14:06:54.355+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a3b850c0 2021-02-20T14:06:54.361 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T14:06:54.359+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a600e220 2021-02-20T14:06:54.361 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T14:06:54.359+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b5aa55a0 2021-02-20T14:06:54.362 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T14:06:54.359+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b5aa5810 2021-02-20T14:06:54.362 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T14:06:54.359+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b5aa5b50 2021-02-20T14:06:54.362 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T14:06:54.359+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b5aa5c20 2021-02-20T14:06:54.363 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T14:06:54.359+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b66a6820 2021-02-20T14:06:54.363 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T14:06:54.359+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b9ff4f70 2021-02-20T14:06:54.363 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T14:06:54.359+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b9ff5ba0 2021-02-20T14:07:56.770 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5922715702500492) > weight (0.25) 2021-02-20T14:07:56.771 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.29820701734279365) > weight (0.25) 2021-02-20T14:07:56.771 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8236016629333617) > weight (0.25) 2021-02-20T14:07:56.771 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T14:07:56.772 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T14:07:56.772 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T14:07:56.773 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T14:07:56.773+0000 7f370c67a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 35633) UID: 1000 2021-02-20T14:07:56.973 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:08:05.774 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T14:08:05.774 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:08:05.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:08:05.775 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T14:08:05.775 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T14:08:05.776 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T14:08:05.778 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T14:08:05.779 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T14:08:16.084 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3492291945079137) > weight (0.25) 2021-02-20T14:08:16.085 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T14:08:16.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T14:08:16.086 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T14:08:16.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T14:08:16.085+0000 7f22f0031700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 36000) UID: 1000 2021-02-20T14:08:16.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:08:20.087 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T14:08:20.089 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:08:20.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:08:20.090 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T14:08:20.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T14:08:20.091 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T14:08:20.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T14:08:20.094 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T14:09:11.780 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:09:11.789 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:09:11.790 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T14:09:11.790 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T14:09:11.791+0000 7f9f9b648700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 36195) UID: 1000 2021-02-20T14:09:11.991 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:09:15.792 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:09:15.802 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:09:15.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:09:15.803 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:09:15.803 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:09:15.804 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:09:15.815 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:09:15.818 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T14:09:15.987 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T14:09:15.987+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b5aa3e10 2021-02-20T14:09:15.995 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T14:09:15.995+0000 7f35a234f700 -1 rbd::mirror::ImageReplayer: 0x55708901a300 [3/49a0b5eb-7d69-4824-806b-1ece12354eb9] start: already running 2021-02-20T14:09:16.008 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T14:09:16.007+0000 7f35a234f700 -1 rbd::mirror::ImageReplayer: 0x55708cfc1180 [3/945695ad-0f15-4fd7-a092-4e7df53e1d1b] start: already running 2021-02-20T14:09:16.009 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T14:09:16.007+0000 7f35a234f700 -1 rbd::mirror::ImageReplayer: 0x55708cfc1400 [3/972a582c-01df-4309-8926-a24b908ec0fc] start: already running 2021-02-20T14:09:16.009 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T14:09:16.007+0000 7f35a234f700 -1 rbd::mirror::ImageReplayer: 0x55708cfc1680 [3/9940675f-5679-4d59-9726-da9895d6bb7c] start: already running 2021-02-20T14:09:16.010 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T14:09:16.007+0000 7f35a234f700 -1 rbd::mirror::ImageReplayer: 0x55708cfc1900 [3/9b42b7b2-0767-46b8-a6a6-e0b14c0863eb] start: already running 2021-02-20T14:09:16.010 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T14:09:16.007+0000 7f35a234f700 -1 rbd::mirror::ImageReplayer: 0x55708cfc1b80 [3/9bb53f74-14d3-4057-8f64-9bba63f946e2] start: already running 2021-02-20T14:09:16.011 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T14:09:16.007+0000 7f35a234f700 -1 rbd::mirror::ImageReplayer: 0x55708cfc1e00 [3/a3cf9f38-0520-4671-b928-6d907b15b2fb] start: already running 2021-02-20T14:09:16.011 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T14:09:16.007+0000 7f35a234f700 -1 rbd::mirror::ImageReplayer: 0x55708cfc2080 [3/a3d3aab0-3900-425e-a825-80fa9ff092ea] start: already running 2021-02-20T14:10:02.095 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8016657986050636) > weight (0.25) 2021-02-20T14:10:02.097 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.306755156446359) > weight (0.25) 2021-02-20T14:10:02.098 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T14:10:02.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T14:10:02.099 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T14:10:02.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T14:10:02.098+0000 7f5988247700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 35007) UID: 1000 2021-02-20T14:10:02.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:10:06.099 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T14:10:06.099 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:10:06.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:10:06.100 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T14:10:06.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T14:10:06.101 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T14:10:06.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T14:10:06.105 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T14:10:27.819 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2510775743762371) > weight (0.25) 2021-02-20T14:10:27.819 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:10:27.820 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:10:27.820 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T14:10:27.821 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T14:10:27.817+0000 7f2a636c2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 35725) UID: 1000 2021-02-20T14:10:28.022 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:10:29.821 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:10:29.822 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:10:29.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:10:29.823 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:10:29.823 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:10:29.823 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:10:29.832 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:10:29.832 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T14:12:01.107 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5508044986415312) > weight (0.25) 2021-02-20T14:12:01.108 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.46058974732662694) > weight (0.25) 2021-02-20T14:12:01.109 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5078090389936675) > weight (0.25) 2021-02-20T14:12:01.109 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7953193383858226) > weight (0.25) 2021-02-20T14:12:01.109 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T14:12:04.833 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8743243837246999) > weight (0.25) 2021-02-20T14:12:04.834 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:12:04.835 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:12:04.835 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T14:12:04.836 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T14:12:04.835+0000 7f39595bb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 36667) UID: 1000 2021-02-20T14:12:05.036 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:12:10.836 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:12:10.837 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:12:10.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:12:10.838 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:12:10.838 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:12:10.838 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:12:10.841 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:12:10.842 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T14:13:31.110 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7604562948196238) > weight (0.25) 2021-02-20T14:13:31.118 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5463070113114072) > weight (0.25) 2021-02-20T14:13:31.118 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7126509962210894) > weight (0.25) 2021-02-20T14:13:31.119 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8732340549255658) > weight (0.25) 2021-02-20T14:13:31.119 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T14:13:48.842 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8682296786011915) > weight (0.25) 2021-02-20T14:13:48.843 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6393183386165541) > weight (0.25) 2021-02-20T14:13:48.843 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6993598348450241) > weight (0.25) 2021-02-20T14:13:48.843 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8256439228559129) > weight (0.25) 2021-02-20T14:13:48.843 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T14:14:40.036 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T14:14:40.034+0000 7fb8259e4700 -1 reset not still connected to 0x557852507040 2021-02-20T14:14:40.057 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T14:14:40.034+0000 7fb8259e4700 -1 reset not still connected to 0x557866b07810 2021-02-20T14:14:59.844 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3231485705382735) > weight (0.25) 2021-02-20T14:14:59.853 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7197071300944815) > weight (0.25) 2021-02-20T14:14:59.853 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8746055378655446) > weight (0.25) 2021-02-20T14:14:59.854 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5897860241067954) > weight (0.25) 2021-02-20T14:14:59.854 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T14:15:14.120 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5218718978035841) > weight (0.25) 2021-02-20T14:15:14.120 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6313633413636399) > weight (0.25) 2021-02-20T14:15:14.121 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2947744330475458) > weight (0.25) 2021-02-20T14:15:14.121 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7093640356807235) > weight (0.25) 2021-02-20T14:15:14.121 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T14:16:49.122 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6211272416436021) > weight (0.25) 2021-02-20T14:16:49.129 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T14:16:49.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T14:16:49.130 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T14:16:49.130 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T14:16:49.131 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:16:49.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T14:16:49.132+0000 7fec75a02700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 36377) UID: 1000 2021-02-20T14:16:49.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:16:52.855 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6537177933975403) > weight (0.25) 2021-02-20T14:16:52.856 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:16:52.856 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:16:52.857 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T14:16:52.858 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T14:16:52.856+0000 7f3d99c91700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 36760) UID: 1000 2021-02-20T14:16:53.058 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:16:55.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:16:55.233 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T14:16:55.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T14:16:55.233 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T14:16:55.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T14:16:55.236 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T14:16:57.858 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:16:57.858 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:16:57.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:16:57.859 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:16:57.859 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:16:57.859 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:16:57.862 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:16:57.862 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T14:17:57.237 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3968529877781485) > weight (0.25) 2021-02-20T14:17:57.253 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9683876737763469) > weight (0.25) 2021-02-20T14:17:57.255 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5149568075167377) > weight (0.25) 2021-02-20T14:17:57.256 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4895204313669773) > weight (0.25) 2021-02-20T14:17:57.257 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T14:18:30.863 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2605623386503638) > weight (0.25) 2021-02-20T14:18:30.870 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:18:30.871 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:18:30.872 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T14:18:30.873 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T14:18:30.869+0000 7f2c34112700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 36955) UID: 1000 2021-02-20T14:18:31.073 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:18:34.873 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:18:34.873 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:18:34.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:18:34.874 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:18:34.874 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:18:34.874 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:18:34.877 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:18:34.877 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T14:19:35.260 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:19:35.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:19:35.288 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T14:19:35.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T14:19:35.287+0000 7f1edc29f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 35541) UID: 1000 2021-02-20T14:19:35.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:19:39.290 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:19:39.296 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:19:39.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:19:39.297 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:19:39.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:19:39.298 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:19:39.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:19:39.301 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T14:19:53.878 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.33438468523868303) > weight (0.25) 2021-02-20T14:19:53.880 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6451823221902546) > weight (0.25) 2021-02-20T14:19:53.880 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4179150126193354) > weight (0.25) 2021-02-20T14:19:53.880 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6752528386341035) > weight (0.25) 2021-02-20T14:19:53.881 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T14:20:25.067 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T14:20:25.062+0000 7fb8259e4700 -1 reset not still connected to 0x557855c1d5f0 2021-02-20T14:20:25.068 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T14:20:25.062+0000 7fb8259e4700 -1 reset not still connected to 0x557855c1f0c0 2021-02-20T14:20:25.068 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T14:20:25.066+0000 7fb8259e4700 -1 reset not still connected to 0x557865e37670 2021-02-20T14:20:25.070 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T14:20:25.066+0000 7fb8259e4700 -1 reset not still connected to 0x557866b04dd0 2021-02-20T14:20:25.070 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T14:20:25.066+0000 7fb8259e4700 -1 reset not still connected to 0x557866b05ba0 2021-02-20T14:20:25.071 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T14:20:25.066+0000 7fb8259e4700 -1 reset not still connected to 0x557866b07260 2021-02-20T14:20:25.071 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T14:20:25.066+0000 7fb8259e4700 -1 reset not still connected to 0x557866b079b0 2021-02-20T14:21:11.303 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6362329611675048) > weight (0.25) 2021-02-20T14:21:11.304 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T14:21:11.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T14:21:11.306 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T14:21:11.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T14:21:11.308+0000 7f443379d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 36864) UID: 1000 2021-02-20T14:21:11.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:21:12.307 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T14:21:12.307 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:21:12.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:21:12.308 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T14:21:12.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T14:21:12.309 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T14:21:12.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T14:21:12.312 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T14:21:25.882 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7910818528551866) > weight (0.25) 2021-02-20T14:21:25.917 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.37776829341988827) > weight (0.25) 2021-02-20T14:21:25.917 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4524663548258202) > weight (0.25) 2021-02-20T14:21:25.917 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.41027550092211573) > weight (0.25) 2021-02-20T14:21:25.918 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T14:22:30.318 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5647481110100669) > weight (0.25) 2021-02-20T14:22:30.330 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5337415055453851) > weight (0.25) 2021-02-20T14:22:30.330 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5521207309482695) > weight (0.25) 2021-02-20T14:22:30.330 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.2681616217053042) > weight (0.25) 2021-02-20T14:22:30.331 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T14:23:22.919 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5727219042591546) > weight (0.25) 2021-02-20T14:23:22.927 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6103349321698941) > weight (0.25) 2021-02-20T14:23:22.927 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7493701880184747) > weight (0.25) 2021-02-20T14:23:22.927 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T14:23:22.928 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T14:23:22.928 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T14:23:22.930 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T14:23:22.925+0000 7f3f6c8e0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 36286) UID: 1000 2021-02-20T14:23:23.130 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:23:25.929 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T14:23:25.936 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:23:25.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:23:25.937 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T14:23:25.937 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T14:23:25.938 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T14:23:25.947 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T14:23:25.948 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T14:24:06.331 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9850527716827345) > weight (0.25) 2021-02-20T14:24:06.361 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7268879043426736) > weight (0.25) 2021-02-20T14:24:06.361 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7423044237358914) > weight (0.25) 2021-02-20T14:24:06.362 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9800597722441201) > weight (0.25) 2021-02-20T14:24:06.362 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T14:24:36.949 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8028017322944448) > weight (0.25) 2021-02-20T14:24:36.956 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4860145175252143) > weight (0.25) 2021-02-20T14:24:36.956 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6718036492181737) > weight (0.25) 2021-02-20T14:24:36.956 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8481257662291004) > weight (0.25) 2021-02-20T14:24:36.956 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T14:25:44.363 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5774691846031067) > weight (0.25) 2021-02-20T14:25:44.374 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.27728927594945485) > weight (0.25) 2021-02-20T14:25:44.374 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.49677634617418465) > weight (0.25) 2021-02-20T14:25:44.375 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.89862675635962) > weight (0.25) 2021-02-20T14:25:44.375 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T14:26:09.958 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.25791318448558687) > weight (0.25) 2021-02-20T14:26:09.964 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7291350429124079) > weight (0.25) 2021-02-20T14:26:09.965 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.30903212519174716) > weight (0.25) 2021-02-20T14:26:09.965 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5441547776547472) > weight (0.25) 2021-02-20T14:26:09.965 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T14:26:50.378 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T14:26:50.372+0000 7f03ccd38700 -1 reset not still connected to 0x556b8794d110 2021-02-20T14:26:50.378 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T14:26:50.372+0000 7f03ccd38700 -1 reset not still connected to 0x556b8794d380 2021-02-20T14:26:50.379 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T14:26:50.376+0000 7f03ccd38700 -1 reset not still connected to 0x556b8f7beea0 2021-02-20T14:26:50.380 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T14:26:50.376+0000 7f03ccd38700 -1 reset not still connected to 0x556b94ec50c0 2021-02-20T14:26:50.380 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T14:26:50.376+0000 7f03ccd38700 -1 reset not still connected to 0x556b9f07e080 2021-02-20T14:27:33.375 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:27:33.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:27:33.418 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T14:27:33.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T14:27:33.418+0000 7f6d860c7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 37141) UID: 1000 2021-02-20T14:27:33.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:27:35.419 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:27:35.420 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:27:35.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:27:35.421 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:27:35.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:27:35.422 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:27:35.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:27:35.424 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T14:27:49.967 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:27:49.973 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:27:49.974 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T14:27:49.975 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T14:27:49.974+0000 7f9f4d7f0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 36476) UID: 1000 2021-02-20T14:27:50.175 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:27:55.975 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:27:55.982 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:27:55.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:27:55.983 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:27:55.983 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:27:55.983 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:27:55.986 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:27:55.986 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T14:28:56.426 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:28:56.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:28:56.436 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T14:28:56.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T14:28:56.436+0000 7f6d099c0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 37422) UID: 1000 2021-02-20T14:28:56.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:28:57.437 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:28:57.438 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:28:57.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:28:57.438 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:28:57.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:28:57.439 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:28:57.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:28:57.442 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T14:29:26.987 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8541236925390454) > weight (0.25) 2021-02-20T14:29:27.023 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:29:27.024 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:29:27.024 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T14:29:27.025 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T14:29:27.023+0000 7f7930956700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 37050) UID: 1000 2021-02-20T14:29:27.225 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:29:31.025 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:29:31.026 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:29:31.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:29:31.026 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:29:31.027 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:29:31.027 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:29:31.030 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:29:31.030 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T14:30:33.031 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:30:33.040 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:30:33.041 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T14:30:33.043 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T14:30:33.038+0000 7f5e7132c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 37514) UID: 1000 2021-02-20T14:30:33.242 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:30:35.443 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:30:35.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:30:35.445 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T14:30:35.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T14:30:35.442+0000 7f58d688d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 37605) UID: 1000 2021-02-20T14:30:35.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:30:41.042 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:30:41.042 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:30:41.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:30:41.043 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:30:41.043 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:30:41.044 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:30:41.046 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:30:41.047 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T14:30:43.446 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:30:43.446 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:30:43.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:30:43.447 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:30:43.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:30:43.448 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:30:43.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:30:43.451 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T14:32:17.047 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9999443844739829) > weight (0.25) 2021-02-20T14:32:17.084 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:32:17.085 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:32:17.085 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T14:32:17.086 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T14:32:17.083+0000 7f52a8b95700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 37697) UID: 1000 2021-02-20T14:32:17.286 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:32:25.086 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:32:25.093 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:32:25.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:32:25.094 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:32:25.095 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:32:25.095 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:32:25.098 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:32:25.098 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T14:32:39.453 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6533730968271996) > weight (0.25) 2021-02-20T14:32:39.460 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5828227887564528) > weight (0.25) 2021-02-20T14:32:39.460 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T14:32:39.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T14:32:39.461 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T14:32:39.461 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T14:32:39.462 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:32:39.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T14:32:39.458+0000 7fad42d1e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 36568) UID: 1000 2021-02-20T14:32:39.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:32:45.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:32:45.573 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T14:32:45.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T14:32:45.574 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T14:32:45.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T14:32:45.577 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T14:33:46.099 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.40500078879698487) > weight (0.25) 2021-02-20T14:33:46.103 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.49278536864542055) > weight (0.25) 2021-02-20T14:33:46.103 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9536948101360843) > weight (0.25) 2021-02-20T14:33:46.103 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6653644904708926) > weight (0.25) 2021-02-20T14:33:46.104 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T14:34:31.578 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.28996386031107635) > weight (0.25) 2021-02-20T14:34:31.618 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T14:34:31.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T14:34:31.619 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T14:34:31.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T14:34:31.619+0000 7fd435cd1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 37234) UID: 1000 2021-02-20T14:34:31.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:34:36.620 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T14:34:36.621 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:34:36.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:34:36.622 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T14:34:36.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T14:34:36.622 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T14:34:36.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T14:34:36.625 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T14:35:21.104 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:35:21.117 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:35:21.118 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T14:35:21.119 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T14:35:21.118+0000 7f96502f8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 37791) UID: 1000 2021-02-20T14:35:21.320 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:35:23.119 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:35:23.120 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:35:23.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:35:23.120 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:35:23.121 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:35:23.121 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:35:23.124 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:35:23.124 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T14:35:54.626 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:35:54.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:35:54.635 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T14:35:54.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T14:35:54.637+0000 7f241bd24700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 37882) UID: 1000 2021-02-20T14:35:54.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:35:56.636 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:35:56.637 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:35:56.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:35:56.638 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:35:56.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:35:56.638 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:35:56.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:35:56.642 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T14:36:36.125 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.318843383369267) > weight (0.25) 2021-02-20T14:36:36.161 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8670062229678143) > weight (0.25) 2021-02-20T14:36:36.162 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6579947844595435) > weight (0.25) 2021-02-20T14:36:36.162 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8739722158687457) > weight (0.25) 2021-02-20T14:36:36.162 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T14:37:19.642 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7407649024077091) > weight (0.25) 2021-02-20T14:37:19.649 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T14:37:19.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T14:37:19.650 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T14:37:19.650 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T14:37:19.651 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:37:19.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T14:37:19.651+0000 7f31aed6a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 38156) UID: 1000 2021-02-20T14:37:19.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:37:25.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:37:25.753 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T14:37:25.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T14:37:25.754 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T14:37:25.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T14:37:25.757 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T14:37:46.163 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:37:46.165 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:37:46.165 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T14:37:46.166 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T14:37:46.166+0000 7f93de183700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 38251) UID: 1000 2021-02-20T14:37:46.366 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:37:53.165 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:37:53.166 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:37:53.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:37:53.168 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:37:53.168 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:37:53.169 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:37:53.173 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:37:53.173 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T14:38:40.758 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7003973010984059) > weight (0.25) 2021-02-20T14:38:40.766 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.30798758844847707) > weight (0.25) 2021-02-20T14:38:40.766 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7738356729782337) > weight (0.25) 2021-02-20T14:38:40.766 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T14:38:40.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T14:38:40.768 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T14:38:40.768 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T14:38:40.768 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:38:40.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T14:38:40.765+0000 7f438c60d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 36104) UID: 1000 2021-02-20T14:38:41.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:38:46.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:38:46.877 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T14:38:46.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T14:38:46.878 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T14:38:46.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T14:38:46.881 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T14:39:24.176 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9022525901580317) > weight (0.25) 2021-02-20T14:39:24.195 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9904232380537539) > weight (0.25) 2021-02-20T14:39:24.195 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T14:39:24.196 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T14:39:24.196 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T14:39:24.198 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T14:39:24.196+0000 7f35a2b50700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 35816) UID: 1000 2021-02-20T14:39:24.398 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:39:28.197 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T14:39:28.198 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:39:28.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:39:28.199 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T14:39:28.199 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T14:39:28.200 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T14:39:28.202 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T14:39:28.203 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T14:39:28.479 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T14:39:28.476+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ac75cb10 2021-02-20T14:39:28.479 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T14:39:28.476+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6e58accb0 2021-02-20T14:39:28.480 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T14:39:28.476+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6e58ad400 2021-02-20T14:40:28.882 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6713169056415312) > weight (0.25) 2021-02-20T14:40:28.888 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.46606725847811736) > weight (0.25) 2021-02-20T14:40:28.888 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T14:40:28.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T14:40:28.889 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T14:40:28.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T14:40:28.890+0000 7f22c002a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 38064) UID: 1000 2021-02-20T14:40:29.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:40:29.896 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T14:40:29.897 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:40:29.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:40:29.897 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T14:40:29.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T14:40:29.898 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T14:40:29.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T14:40:29.902 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T14:40:33.203 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:40:33.204 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:40:33.204 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T14:40:33.205 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T14:40:33.206+0000 7fd3a2f13700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 38526) UID: 1000 2021-02-20T14:40:33.406 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:40:38.205 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:40:38.206 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:40:38.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:40:38.207 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:40:38.207 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:40:38.207 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:40:38.210 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:40:38.210 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T14:40:38.554 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T14:40:38.550+0000 7f03ccd38700 -1 reset not still connected to 0x556b7ff89330 2021-02-20T14:40:38.554 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T14:40:38.550+0000 7f03ccd38700 -1 reset not still connected to 0x556b8794f330 2021-02-20T14:41:51.903 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.272008591535822) > weight (0.25) 2021-02-20T14:41:51.911 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T14:41:51.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T14:41:51.913 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T14:41:51.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T14:41:51.912+0000 7f504b327700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 38435) UID: 1000 2021-02-20T14:41:52.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:41:53.913 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T14:41:53.917 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:41:53.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:41:53.918 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T14:41:53.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T14:41:53.919 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T14:41:53.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T14:41:53.922 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T14:42:19.211 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7514861054239381) > weight (0.25) 2021-02-20T14:42:19.219 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:42:19.220 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:42:19.221 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T14:42:19.222 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T14:42:19.219+0000 7f3ae0a68700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 37973) UID: 1000 2021-02-20T14:42:19.422 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:42:23.221 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:42:23.222 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:42:23.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:42:23.223 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:42:23.223 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:42:23.223 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:42:23.226 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:42:23.227 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T14:43:40.924 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6298317393850335) > weight (0.25) 2021-02-20T14:43:40.931 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.26652780680754795) > weight (0.25) 2021-02-20T14:43:40.931 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9634271355358334) > weight (0.25) 2021-02-20T14:43:40.932 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5922646666028732) > weight (0.25) 2021-02-20T14:43:40.932 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T14:44:13.228 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:44:13.229 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:44:13.230 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T14:44:13.231 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T14:44:13.232+0000 7fc3e84b0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 38900) UID: 1000 2021-02-20T14:44:13.431 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:44:14.230 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:44:14.231 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:44:14.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:44:14.232 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:44:14.232 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:44:14.232 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:44:14.235 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:44:14.235 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T14:45:11.933 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8852568753901129) > weight (0.25) 2021-02-20T14:45:11.934 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7717866974177823) > weight (0.25) 2021-02-20T14:45:11.934 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8463565004596534) > weight (0.25) 2021-02-20T14:45:11.934 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3837092816986982) > weight (0.25) 2021-02-20T14:45:11.934 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T14:46:02.237 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5376018166528176) > weight (0.25) 2021-02-20T14:46:02.238 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:46:02.239 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:46:02.239 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T14:46:02.240 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T14:46:02.237+0000 7f5b90341700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 39084) UID: 1000 2021-02-20T14:46:02.440 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:46:04.240 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:46:04.241 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:46:04.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:46:04.242 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:46:04.242 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:46:04.242 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:46:04.245 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:46:04.246 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T14:46:29.936 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.25260341861010394) > weight (0.25) 2021-02-20T14:46:29.937 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8530565381945274) > weight (0.25) 2021-02-20T14:46:29.937 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6902806492861279) > weight (0.25) 2021-02-20T14:46:29.937 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3173431603841216) > weight (0.25) 2021-02-20T14:46:29.938 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T14:47:09.248 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.937919757641855) > weight (0.25) 2021-02-20T14:47:09.249 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2911548216254606) > weight (0.25) 2021-02-20T14:47:09.249 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7124709211706304) > weight (0.25) 2021-02-20T14:47:09.249 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8390327872981271) > weight (0.25) 2021-02-20T14:47:09.249 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T14:48:03.939 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7450445245139479) > weight (0.25) 2021-02-20T14:48:03.950 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7825743772065977) > weight (0.25) 2021-02-20T14:48:03.951 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9267468696911365) > weight (0.25) 2021-02-20T14:48:03.952 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9272865434259364) > weight (0.25) 2021-02-20T14:48:03.953 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T14:48:52.251 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6840604681490267) > weight (0.25) 2021-02-20T14:48:52.260 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5341360091448314) > weight (0.25) 2021-02-20T14:48:52.261 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.45090573125378086) > weight (0.25) 2021-02-20T14:48:52.261 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T14:48:52.262 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T14:48:52.262 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T14:48:52.263 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T14:48:52.263 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:48:52.264 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T14:48:52.265+0000 7f564d708700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 37326) UID: 1000 2021-02-20T14:48:52.464 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:48:58.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:48:58.366 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T14:48:58.366 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T14:48:58.367 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T14:48:58.373 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T14:48:58.374 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T14:49:13.954 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6326457200515578) > weight (0.25) 2021-02-20T14:49:13.962 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3321793632978003) > weight (0.25) 2021-02-20T14:49:13.962 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.802705147897951) > weight (0.25) 2021-02-20T14:49:13.962 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.699377037970991) > weight (0.25) 2021-02-20T14:49:13.963 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T14:50:19.376 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6195419749009383) > weight (0.25) 2021-02-20T14:50:19.384 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9294682384859183) > weight (0.25) 2021-02-20T14:50:19.384 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7725960978952743) > weight (0.25) 2021-02-20T14:50:19.384 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.692692177138422) > weight (0.25) 2021-02-20T14:50:19.385 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T14:51:00.964 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5379725039486714) > weight (0.25) 2021-02-20T14:51:00.972 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8480403298352498) > weight (0.25) 2021-02-20T14:51:00.972 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8410460939532695) > weight (0.25) 2021-02-20T14:51:00.973 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.49238027634794423) > weight (0.25) 2021-02-20T14:51:00.973 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T14:51:59.386 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7461986219933201) > weight (0.25) 2021-02-20T14:51:59.387 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4232810466337934) > weight (0.25) 2021-02-20T14:51:59.387 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3159588113855146) > weight (0.25) 2021-02-20T14:51:59.387 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T14:51:59.388 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T14:51:59.389 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T14:51:59.389 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T14:51:59.389 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:51:59.392 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T14:51:59.392+0000 7fe143c2d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 39364) UID: 1000 2021-02-20T14:51:59.592 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:52:05.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:52:05.492 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T14:52:05.493 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T14:52:05.493 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T14:52:05.496 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T14:52:05.497 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T14:52:31.974 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:52:31.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:52:31.975 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T14:52:31.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T14:52:31.975+0000 7fe2cd9ac700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 38343) UID: 1000 2021-02-20T14:52:32.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:52:34.976 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:52:34.976 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:52:34.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:52:34.977 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:52:34.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:52:34.978 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:52:34.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:52:34.981 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T14:52:36.626 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T14:52:36.623+0000 7fb8259e4700 -1 reset not still connected to 0x557855c1e3c0 2021-02-20T14:52:36.627 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T14:52:36.623+0000 7fb8259e4700 -1 reset not still connected to 0x557860c124e0 2021-02-20T14:52:36.627 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T14:52:36.623+0000 7fb8259e4700 -1 reset not still connected to 0x557860c13860 2021-02-20T14:53:20.497 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:53:20.499 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:53:20.499 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T14:53:20.500 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T14:53:20.498+0000 7f73c054f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 39176) UID: 1000 2021-02-20T14:53:20.700 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:53:28.500 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:53:28.500 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:53:28.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:53:28.501 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:53:28.502 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:53:28.502 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:53:28.505 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:53:28.506 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T14:53:49.983 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8762881146454194) > weight (0.25) 2021-02-20T14:53:49.983 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3946007540339822) > weight (0.25) 2021-02-20T14:53:49.983 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9090344350674124) > weight (0.25) 2021-02-20T14:53:49.984 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6252538129238114) > weight (0.25) 2021-02-20T14:53:49.984 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T14:54:55.507 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3935503354398957) > weight (0.25) 2021-02-20T14:54:55.508 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7138221524655319) > weight (0.25) 2021-02-20T14:54:55.508 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.41319641289126785) > weight (0.25) 2021-02-20T14:54:55.508 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.254979405221866) > weight (0.25) 2021-02-20T14:54:55.509 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T14:55:20.985 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.32674245112367684) > weight (0.25) 2021-02-20T14:55:20.985 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.34719556773142757) > weight (0.25) 2021-02-20T14:55:20.985 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9669209307448766) > weight (0.25) 2021-02-20T14:55:20.986 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.43687091243934295) > weight (0.25) 2021-02-20T14:55:20.986 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T14:56:54.509 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6841548032586432) > weight (0.25) 2021-02-20T14:56:54.510 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:56:54.511 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:56:54.511 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T14:56:54.512 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T14:56:54.512+0000 7f5c8968a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 39272) UID: 1000 2021-02-20T14:56:54.713 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:57:00.513 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:57:00.513 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:57:00.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:57:00.514 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:57:00.514 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:57:00.515 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:57:00.518 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:57:00.518 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T14:57:09.987 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.40848101646651525) > weight (0.25) 2021-02-20T14:57:09.987 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.39964325219406294) > weight (0.25) 2021-02-20T14:57:09.988 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6684941325813217) > weight (0.25) 2021-02-20T14:57:09.988 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8248674234164947) > weight (0.25) 2021-02-20T14:57:09.988 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T14:58:10.988 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:58:10.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:58:10.990 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T14:58:10.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T14:58:10.990+0000 7f52b3de4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 39550) UID: 1000 2021-02-20T14:58:11.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:58:11.991 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:58:11.991 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:58:11.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:58:11.992 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:58:11.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:58:11.993 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:58:11.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:58:11.995 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T14:58:15.520 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9233948686005831) > weight (0.25) 2021-02-20T14:58:15.520 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9097828511658608) > weight (0.25) 2021-02-20T14:58:15.521 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9370057105348503) > weight (0.25) 2021-02-20T14:58:15.521 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6955057364915166) > weight (0.25) 2021-02-20T14:58:15.522 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T14:59:36.996 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.39468302108352527) > weight (0.25) 2021-02-20T14:59:37.005 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9213920042242597) > weight (0.25) 2021-02-20T14:59:37.006 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6317323937979353) > weight (0.25) 2021-02-20T14:59:37.006 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T14:59:37.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T14:59:37.007 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T14:59:37.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T14:59:37.007+0000 7fbda0d74700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 38625) UID: 1000 2021-02-20T14:59:37.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:59:42.008 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T14:59:42.009 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:59:42.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:59:42.009 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T14:59:42.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T14:59:42.010 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T14:59:42.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T14:59:42.017 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T15:00:13.524 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6464031533511465) > weight (0.25) 2021-02-20T15:00:13.531 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6389626225338614) > weight (0.25) 2021-02-20T15:00:13.531 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9860361717238897) > weight (0.25) 2021-02-20T15:00:13.532 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5473331315053267) > weight (0.25) 2021-02-20T15:00:13.532 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T15:01:14.018 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3366218250973204) > weight (0.25) 2021-02-20T15:01:14.027 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T15:01:14.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T15:01:14.028 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T15:01:14.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T15:01:14.029+0000 7f1e06cfb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 38993) UID: 1000 2021-02-20T15:01:14.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:01:18.029 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T15:01:18.030 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:01:18.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:01:18.031 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T15:01:18.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T15:01:18.032 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T15:01:18.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T15:01:18.035 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T15:02:05.533 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:02:05.535 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:02:05.535 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T15:02:05.536 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T15:02:05.535+0000 7f0d9e985700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 39649) UID: 1000 2021-02-20T15:02:05.736 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:02:14.536 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:02:14.537 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:02:14.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:02:14.538 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:02:14.538 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:02:14.539 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:02:14.541 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:02:14.542 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T15:02:58.036 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9213328263657015) > weight (0.25) 2021-02-20T15:02:58.037 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2871343554220024) > weight (0.25) 2021-02-20T15:02:58.037 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T15:02:58.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T15:02:58.038 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T15:02:58.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T15:02:58.038+0000 7f65d1649700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 38808) UID: 1000 2021-02-20T15:02:58.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:03:07.039 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T15:03:07.039 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:03:07.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:03:07.040 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T15:03:07.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T15:03:07.041 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T15:03:07.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T15:03:07.046 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T15:04:10.543 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5678078964207423) > weight (0.25) 2021-02-20T15:04:10.544 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.811142856367586) > weight (0.25) 2021-02-20T15:04:10.544 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3988884925221753) > weight (0.25) 2021-02-20T15:04:10.544 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8597455786151319) > weight (0.25) 2021-02-20T15:04:10.544 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T15:05:01.048 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:05:01.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:05:01.056 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T15:05:01.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T15:05:01.055+0000 7f5b901bb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 39837) UID: 1000 2021-02-20T15:05:01.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:05:02.056 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:05:02.057 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:05:02.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:05:02.057 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:05:02.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:05:02.058 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:05:02.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:05:02.061 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T15:05:10.545 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7380770126701519) > weight (0.25) 2021-02-20T15:05:10.546 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3998732279066475) > weight (0.25) 2021-02-20T15:05:10.546 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9812265673710352) > weight (0.25) 2021-02-20T15:05:10.546 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.26911358224292903) > weight (0.25) 2021-02-20T15:05:10.547 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T15:06:20.062 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6159983161327169) > weight (0.25) 2021-02-20T15:06:20.072 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9322229855062133) > weight (0.25) 2021-02-20T15:06:20.072 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4785197156171045) > weight (0.25) 2021-02-20T15:06:20.072 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7531547022182042) > weight (0.25) 2021-02-20T15:06:20.073 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T15:06:45.547 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.844796495814819) > weight (0.25) 2021-02-20T15:06:45.552 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9290447803429318) > weight (0.25) 2021-02-20T15:06:45.552 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6126253052195575) > weight (0.25) 2021-02-20T15:06:45.553 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.754199570453555) > weight (0.25) 2021-02-20T15:06:45.553 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T15:07:44.074 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:07:44.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:07:44.076 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:07:44.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T15:07:44.074+0000 7f69f1e30700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 40308) UID: 1000 2021-02-20T15:07:44.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:07:52.077 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:07:52.077 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:07:52.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:07:52.078 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:07:52.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:07:52.079 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:07:52.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:07:52.082 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T15:07:55.554 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9695339619749991) > weight (0.25) 2021-02-20T15:07:55.555 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3640368853832602) > weight (0.25) 2021-02-20T15:07:55.555 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9148794852982831) > weight (0.25) 2021-02-20T15:07:55.555 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.783885310687472) > weight (0.25) 2021-02-20T15:07:55.556 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T15:09:33.558 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8180126367138524) > weight (0.25) 2021-02-20T15:09:33.573 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4616587823413203) > weight (0.25) 2021-02-20T15:09:33.574 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7561678300817557) > weight (0.25) 2021-02-20T15:09:33.574 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.27281414711491214) > weight (0.25) 2021-02-20T15:09:33.574 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T15:09:43.083 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8992709436511427) > weight (0.25) 2021-02-20T15:09:43.086 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6571718364681175) > weight (0.25) 2021-02-20T15:09:43.087 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2994946211195815) > weight (0.25) 2021-02-20T15:09:43.087 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.676120301555327) > weight (0.25) 2021-02-20T15:09:43.087 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T15:11:06.088 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6265629003601848) > weight (0.25) 2021-02-20T15:11:06.098 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.31932484880879575) > weight (0.25) 2021-02-20T15:11:06.099 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T15:11:06.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T15:11:06.100 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T15:11:06.101 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T15:11:06.101 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:11:06.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T15:11:06.101+0000 7faa36d80700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 40214) UID: 1000 2021-02-20T15:11:06.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:11:10.575 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.46233202835723985) > weight (0.25) 2021-02-20T15:11:10.575 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5062786819629469) > weight (0.25) 2021-02-20T15:11:10.576 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9021024504156743) > weight (0.25) 2021-02-20T15:11:10.576 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7714857421560325) > weight (0.25) 2021-02-20T15:11:10.576 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T15:11:12.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:11:12.203 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T15:11:12.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T15:11:12.204 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T15:11:12.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T15:11:12.207 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T15:13:05.578 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:13:05.588 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:13:05.588 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:13:05.589 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T15:13:05.590+0000 7fbe4fe64700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 40115) UID: 1000 2021-02-20T15:13:05.789 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:13:09.208 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6573716680436332) > weight (0.25) 2021-02-20T15:13:09.208 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5615012116350817) > weight (0.25) 2021-02-20T15:13:09.209 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8983764355118118) > weight (0.25) 2021-02-20T15:13:09.209 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.637913860995267) > weight (0.25) 2021-02-20T15:13:09.209 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T15:13:13.589 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:13:13.589 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:13:13.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:13:13.590 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:13:13.591 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:13:13.591 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:13:13.594 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:13:13.594 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T15:14:23.595 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:14:23.597 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:14:23.597 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:14:23.598 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T15:14:23.596+0000 7f51b39d6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 40588) UID: 1000 2021-02-20T15:14:23.799 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:14:31.598 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:14:31.599 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:14:31.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:14:31.600 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:14:31.600 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:14:31.601 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:14:31.604 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:14:31.604 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T15:15:04.211 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:15:04.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:15:04.212 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T15:15:04.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T15:15:04.211+0000 7ff3c1ab0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 40401) UID: 1000 2021-02-20T15:15:04.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:15:08.213 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:15:08.214 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:15:08.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:15:08.214 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:15:08.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:15:08.215 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:15:08.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:15:08.218 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T15:16:29.606 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:16:29.607 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:16:29.608 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:16:29.609 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T15:16:29.608+0000 7f2c81c73700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 40679) UID: 1000 2021-02-20T15:16:29.809 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:16:30.219 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.805853386886596) > weight (0.25) 2021-02-20T15:16:30.220 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7393221827270692) > weight (0.25) 2021-02-20T15:16:30.220 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8453365989683798) > weight (0.25) 2021-02-20T15:16:30.220 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T15:16:30.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T15:16:30.221 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T15:16:30.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T15:16:30.220+0000 7f23e4cee700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 39928) UID: 1000 2021-02-20T15:16:30.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:16:37.222 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T15:16:37.222 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:16:37.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:16:37.223 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T15:16:37.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T15:16:37.224 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T15:16:37.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T15:16:37.227 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T15:16:37.608 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:16:37.609 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:16:37.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:16:37.609 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:16:37.610 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:16:37.610 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:16:37.612 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:16:37.613 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T15:17:54.614 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6851159989645975) > weight (0.25) 2021-02-20T15:17:54.615 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.43359361453007583) > weight (0.25) 2021-02-20T15:17:54.615 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.30756498709516644) > weight (0.25) 2021-02-20T15:17:54.615 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7138667082119138) > weight (0.25) 2021-02-20T15:17:54.616 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T15:18:27.228 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.673646874772154) > weight (0.25) 2021-02-20T15:18:27.229 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T15:18:27.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T15:18:27.231 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T15:18:27.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T15:18:27.229+0000 7fc94defd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 40024) UID: 1000 2021-02-20T15:18:27.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:18:32.231 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T15:18:32.232 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:18:32.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:18:32.233 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T15:18:32.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T15:18:32.234 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T15:18:32.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T15:18:32.239 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T15:19:04.618 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7997629535503669) > weight (0.25) 2021-02-20T15:19:04.619 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T15:19:04.619 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T15:19:04.620 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T15:19:04.620 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T15:19:04.620+0000 7f08ec83b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 39746) UID: 1000 2021-02-20T15:19:04.821 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:19:05.620 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T15:19:05.621 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:19:05.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:19:05.622 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T15:19:05.622 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T15:19:05.622 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T15:19:05.625 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T15:19:05.625 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T15:20:15.240 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:20:15.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:20:15.252 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:20:15.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T15:20:15.250+0000 7f3083458700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 40773) UID: 1000 2021-02-20T15:20:15.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:20:17.626 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9487354802128944) > weight (0.25) 2021-02-20T15:20:17.626 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4904121810325799) > weight (0.25) 2021-02-20T15:20:17.626 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7994013756862447) > weight (0.25) 2021-02-20T15:20:17.627 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5249836929532236) > weight (0.25) 2021-02-20T15:20:17.627 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T15:20:23.252 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:20:23.260 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:20:23.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:20:23.261 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:20:23.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:20:23.263 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:20:23.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:20:23.266 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T15:21:40.267 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:21:40.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:21:40.279 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:21:40.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T15:21:40.280+0000 7fc653c5c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 41250) UID: 1000 2021-02-20T15:21:40.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:21:48.281 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:21:48.287 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:21:48.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:21:48.288 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:21:48.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:21:48.289 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:21:48.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:21:48.292 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T15:21:54.628 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6647476979210339) > weight (0.25) 2021-02-20T15:21:54.628 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7176021754802914) > weight (0.25) 2021-02-20T15:21:54.629 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T15:21:54.629 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T15:21:54.629 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:21:54.630 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T15:21:54.632+0000 7fa551bb5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 38717) UID: 1000 2021-02-20T15:21:54.831 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:22:02.630 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T15:22:02.631 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:22:02.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:22:02.632 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T15:22:02.632 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T15:22:02.632 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T15:22:02.635 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T15:22:02.635 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T15:22:56.294 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.698118217190249) > weight (0.25) 2021-02-20T15:22:56.302 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6949441607875988) > weight (0.25) 2021-02-20T15:22:56.302 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9957788213371342) > weight (0.25) 2021-02-20T15:22:56.302 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T15:22:56.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T15:22:56.304 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T15:22:56.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T15:22:56.302+0000 7fcbd389b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 40865) UID: 1000 2021-02-20T15:22:56.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:23:00.304 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T15:23:00.307 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:23:00.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:23:00.307 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T15:23:00.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T15:23:00.308 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T15:23:00.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T15:23:00.311 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T15:23:09.636 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2937720321806573) > weight (0.25) 2021-02-20T15:23:09.636 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T15:23:09.637 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T15:23:09.637 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T15:23:09.638 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T15:23:09.638+0000 7f57bb14a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 41150) UID: 1000 2021-02-20T15:23:09.839 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:23:14.638 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T15:23:14.639 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:23:14.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:23:14.640 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T15:23:14.640 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T15:23:14.640 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T15:23:14.644 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T15:23:14.644 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T15:24:14.312 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.39629543460828476) > weight (0.25) 2021-02-20T15:24:14.313 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8761264985449403) > weight (0.25) 2021-02-20T15:24:14.313 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.46984484745027777) > weight (0.25) 2021-02-20T15:24:14.313 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4884122022057662) > weight (0.25) 2021-02-20T15:24:14.313 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T15:25:02.646 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.31950466556867385) > weight (0.25) 2021-02-20T15:25:02.646 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T15:25:02.647 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T15:25:02.648 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T15:25:02.649 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T15:25:02.647+0000 7f002b96f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 41624) UID: 1000 2021-02-20T15:25:02.849 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:25:03.648 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T15:25:03.649 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:25:03.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:25:03.650 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T15:25:03.650 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T15:25:03.650 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T15:25:03.653 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T15:25:03.654 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T15:25:40.314 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4333125461491495) > weight (0.25) 2021-02-20T15:25:40.315 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7416287501726022) > weight (0.25) 2021-02-20T15:25:40.315 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7485257297965948) > weight (0.25) 2021-02-20T15:25:40.316 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4665390782962703) > weight (0.25) 2021-02-20T15:25:40.316 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T15:26:26.655 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.38780300088595454) > weight (0.25) 2021-02-20T15:26:26.656 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8347166547787216) > weight (0.25) 2021-02-20T15:26:26.656 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8864842679772406) > weight (0.25) 2021-02-20T15:26:26.656 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T15:26:26.657 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T15:26:26.657 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T15:26:26.658 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T15:26:26.658 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:26:26.659 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T15:26:26.661+0000 7f9d72ada700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 39459) UID: 1000 2021-02-20T15:26:26.858 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:26:32.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:26:32.760 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T15:26:32.760 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T15:26:32.761 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T15:26:32.764 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T15:26:32.764 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T15:26:42.318 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3949874504652996) > weight (0.25) 2021-02-20T15:26:42.318 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T15:26:42.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T15:26:42.319 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T15:26:42.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T15:26:42.320+0000 7f4913600700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 41059) UID: 1000 2021-02-20T15:26:42.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:26:47.320 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T15:26:47.320 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:26:47.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:26:47.321 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T15:26:47.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T15:26:47.322 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T15:26:47.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T15:26:47.325 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T15:28:06.765 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8361469561205125) > weight (0.25) 2021-02-20T15:28:06.766 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7650860939289794) > weight (0.25) 2021-02-20T15:28:06.766 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6716710891250278) > weight (0.25) 2021-02-20T15:28:06.766 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7054897381989244) > weight (0.25) 2021-02-20T15:28:06.766 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T15:28:36.326 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.38142453303499824) > weight (0.25) 2021-02-20T15:28:36.327 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4067558164722451) > weight (0.25) 2021-02-20T15:28:36.327 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3121588148846455) > weight (0.25) 2021-02-20T15:28:36.327 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7305681289873981) > weight (0.25) 2021-02-20T15:28:36.328 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T15:29:12.768 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8090742732418498) > weight (0.25) 2021-02-20T15:29:12.768 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.653737295440632) > weight (0.25) 2021-02-20T15:29:12.768 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T15:29:12.769 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T15:29:12.770 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T15:29:12.771 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T15:29:12.772+0000 7f5f2ca36700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 41433) UID: 1000 2021-02-20T15:29:12.971 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:29:16.771 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T15:29:16.772 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:29:16.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:29:16.773 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T15:29:16.773 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T15:29:16.773 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T15:29:16.776 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T15:29:16.777 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T15:30:16.328 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5490137840336977) > weight (0.25) 2021-02-20T15:30:16.329 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3531222770253001) > weight (0.25) 2021-02-20T15:30:16.329 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9965892146460957) > weight (0.25) 2021-02-20T15:30:16.330 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T15:30:16.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T15:30:16.331 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T15:30:16.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T15:30:16.331+0000 7fab75db2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 41525) UID: 1000 2021-02-20T15:30:16.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:30:20.332 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T15:30:20.333 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:30:20.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:30:20.334 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T15:30:20.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T15:30:20.335 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T15:30:20.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T15:30:20.337 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T15:30:42.778 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.31370948426372813) > weight (0.25) 2021-02-20T15:30:42.778 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.840536409468584) > weight (0.25) 2021-02-20T15:30:42.779 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5836550058588372) > weight (0.25) 2021-02-20T15:30:42.779 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9732396806084566) > weight (0.25) 2021-02-20T15:30:42.779 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T15:31:26.339 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9074692539791398) > weight (0.25) 2021-02-20T15:31:26.339 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8889497971624328) > weight (0.25) 2021-02-20T15:31:26.340 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3231834813339186) > weight (0.25) 2021-02-20T15:31:26.340 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8474595342854316) > weight (0.25) 2021-02-20T15:31:26.341 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T15:32:06.780 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:32:06.781 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:32:06.782 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T15:32:06.783 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T15:32:06.784+0000 7f904b19e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 40954) UID: 1000 2021-02-20T15:32:06.983 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:32:11.783 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:32:11.783 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:32:11.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:32:11.784 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:32:11.784 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:32:11.785 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:32:11.788 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:32:11.788 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T15:32:59.342 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:32:59.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:32:59.344 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T15:32:59.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T15:32:59.342+0000 7fbe1d9e6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 41342) UID: 1000 2021-02-20T15:32:59.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:33:01.345 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:33:01.345 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:33:01.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:33:01.346 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:33:01.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:33:01.347 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:33:01.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:33:01.350 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T15:33:13.789 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3139420451958722) > weight (0.25) 2021-02-20T15:33:13.789 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4961865190828889) > weight (0.25) 2021-02-20T15:33:13.790 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T15:33:13.790 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T15:33:13.791 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T15:33:13.791 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T15:33:13.790+0000 7f64d24b3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 41995) UID: 1000 2021-02-20T15:33:13.992 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:33:14.792 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T15:33:14.792 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:33:14.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:33:14.793 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T15:33:14.793 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T15:33:14.794 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T15:33:14.796 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T15:33:14.796 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T15:34:20.798 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2827438244927116) > weight (0.25) 2021-02-20T15:34:20.799 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.325626556264483) > weight (0.25) 2021-02-20T15:34:20.799 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7639538763051041) > weight (0.25) 2021-02-20T15:34:20.799 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T15:34:20.800 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T15:34:20.800 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T15:34:20.802 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T15:34:20.800+0000 7f09f13e4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 41810) UID: 1000 2021-02-20T15:34:21.002 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:34:22.802 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T15:34:22.802 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:34:22.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:34:22.804 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T15:34:22.804 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T15:34:22.805 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T15:34:22.808 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T15:34:22.809 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T15:34:28.351 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5600500832571086) > weight (0.25) 2021-02-20T15:34:28.352 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T15:34:28.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T15:34:28.352 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T15:34:28.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T15:34:28.352+0000 7fdaccd1a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 41903) UID: 1000 2021-02-20T15:34:28.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:34:33.354 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T15:34:33.354 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:34:33.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:34:33.355 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T15:34:33.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T15:34:33.355 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T15:34:33.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T15:34:33.359 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T15:36:00.360 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:36:00.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:36:00.362 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T15:36:00.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T15:36:00.361+0000 7fb1a2fea700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 42271) UID: 1000 2021-02-20T15:36:00.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:36:05.363 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:36:05.363 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:36:05.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:36:05.364 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:36:05.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:36:05.365 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:36:05.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:36:05.368 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T15:36:11.810 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:36:11.811 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:36:11.812 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T15:36:11.813 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T15:36:11.809+0000 7f280f4b6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 42180) UID: 1000 2021-02-20T15:36:12.013 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:36:12.813 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:36:12.813 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:36:12.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:36:12.814 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:36:12.815 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:36:12.815 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:36:12.818 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:36:12.818 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T15:37:39.819 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8025529285599198) > weight (0.25) 2021-02-20T15:37:39.820 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9154247949218367) > weight (0.25) 2021-02-20T15:37:39.820 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6431875213555627) > weight (0.25) 2021-02-20T15:37:39.820 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T15:37:39.821 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T15:37:39.821 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T15:37:39.829 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T15:37:39.827+0000 7f12f1c90700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 42462) UID: 1000 2021-02-20T15:37:40.029 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:37:45.822 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T15:37:45.822 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:37:45.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:37:45.823 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T15:37:45.824 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T15:37:45.824 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T15:37:45.827 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T15:37:45.827 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T15:37:46.369 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7254599822952225) > weight (0.25) 2021-02-20T15:37:46.369 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5301179066820157) > weight (0.25) 2021-02-20T15:37:46.370 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.37335541060682476) > weight (0.25) 2021-02-20T15:37:46.370 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4347630855979815) > weight (0.25) 2021-02-20T15:37:46.370 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T15:39:07.372 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:39:07.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:39:07.373 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:39:07.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T15:39:07.372+0000 7f3d09eb4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 42647) UID: 1000 2021-02-20T15:39:07.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:39:15.375 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:39:15.375 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:39:15.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:39:15.376 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:39:15.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:39:15.377 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:39:15.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:39:15.380 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T15:39:18.828 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5940282744468716) > weight (0.25) 2021-02-20T15:39:18.829 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8940947034755563) > weight (0.25) 2021-02-20T15:39:18.829 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9855987132998552) > weight (0.25) 2021-02-20T15:39:18.829 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T15:39:18.830 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T15:39:18.830 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T15:39:18.830 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T15:39:18.831 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:39:18.831 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T15:39:18.832+0000 7f6d5126d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 42831) UID: 1000 2021-02-20T15:39:19.031 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:39:24.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:39:24.932 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T15:39:24.932 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T15:39:24.933 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T15:39:24.936 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T15:39:24.936 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T15:40:59.937 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4723068858135183) > weight (0.25) 2021-02-20T15:40:59.938 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7802055337936342) > weight (0.25) 2021-02-20T15:40:59.938 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4542549792441205) > weight (0.25) 2021-02-20T15:40:59.938 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T15:40:59.939 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T15:40:59.939 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T15:40:59.940 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T15:40:59.941+0000 7f2f3abb0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 43015) UID: 1000 2021-02-20T15:41:00.141 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:41:04.380 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6185207299656736) > weight (0.25) 2021-02-20T15:41:04.381 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9817657909819879) > weight (0.25) 2021-02-20T15:41:04.381 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6775070329790579) > weight (0.25) 2021-02-20T15:41:04.381 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9565610861677089) > weight (0.25) 2021-02-20T15:41:04.382 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T15:41:08.940 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T15:41:08.941 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:41:08.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:41:08.941 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T15:41:08.942 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T15:41:08.942 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T15:41:08.945 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T15:41:08.945 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T15:42:19.946 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2970561607636618) > weight (0.25) 2021-02-20T15:42:19.947 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2844649293426559) > weight (0.25) 2021-02-20T15:42:19.947 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T15:42:19.948 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T15:42:19.948 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T15:42:19.949 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T15:42:19.947+0000 7f0b94369700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 42370) UID: 1000 2021-02-20T15:42:20.149 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:42:22.949 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T15:42:22.949 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:42:22.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:42:22.950 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T15:42:22.950 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T15:42:22.951 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T15:42:22.953 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T15:42:22.954 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T15:42:27.382 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:42:27.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:42:27.383 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:42:27.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T15:42:27.383+0000 7fe56abe1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 42924) UID: 1000 2021-02-20T15:42:27.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:42:35.384 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:42:35.385 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:42:35.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:42:35.385 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:42:35.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:42:35.386 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:42:35.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:42:35.390 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T15:43:43.956 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4994674523157412) > weight (0.25) 2021-02-20T15:43:43.957 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4431921215596326) > weight (0.25) 2021-02-20T15:43:43.958 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.46781876795883937) > weight (0.25) 2021-02-20T15:43:43.958 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7682399257618269) > weight (0.25) 2021-02-20T15:43:43.959 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T15:44:13.391 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:44:13.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:44:13.392 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T15:44:13.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T15:44:13.392+0000 7f9e5580c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 43290) UID: 1000 2021-02-20T15:44:13.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:44:15.394 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:44:15.394 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:44:15.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:44:15.395 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:44:15.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:44:15.396 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:44:15.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:44:15.400 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T15:45:23.960 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:45:23.970 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:45:23.972 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T15:45:23.972 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T15:45:23.970+0000 7ff8257ba700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 42738) UID: 1000 2021-02-20T15:45:24.172 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:45:24.972 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:45:24.973 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:45:24.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:45:24.974 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:45:24.974 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:45:24.974 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:45:24.978 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:45:24.978 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T15:45:49.401 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:45:49.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:45:49.410 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T15:45:49.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T15:45:49.405+0000 7f973f79f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 43381) UID: 1000 2021-02-20T15:45:49.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:45:53.410 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:45:53.418 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:45:53.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:45:53.419 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:45:53.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:45:53.420 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:45:53.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:45:53.423 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T15:46:40.979 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6229567317779318) > weight (0.25) 2021-02-20T15:46:40.984 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.25666448595045577) > weight (0.25) 2021-02-20T15:46:40.984 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6223898432088322) > weight (0.25) 2021-02-20T15:46:40.984 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T15:46:40.985 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T15:46:40.985 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T15:46:40.986 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T15:46:40.984+0000 7fe70ec68700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 43106) UID: 1000 2021-02-20T15:46:41.187 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:46:46.986 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T15:46:46.986 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:46:46.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:46:46.987 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T15:46:46.988 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T15:46:46.988 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T15:46:46.991 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T15:46:46.991 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T15:47:05.424 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6512697766613862) > weight (0.25) 2021-02-20T15:47:05.425 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T15:47:05.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T15:47:05.426 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T15:47:05.426 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T15:47:05.426 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:47:05.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T15:47:05.423+0000 7f427eaa5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 42553) UID: 1000 2021-02-20T15:47:05.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:47:11.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:47:11.529 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T15:47:11.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T15:47:11.530 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T15:47:11.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T15:47:11.534 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T15:48:39.993 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.26835732895741626) > weight (0.25) 2021-02-20T15:48:39.994 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6354658841325742) > weight (0.25) 2021-02-20T15:48:39.994 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T15:48:39.995 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T15:48:39.995 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T15:48:39.997 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T15:48:39.997+0000 7f5e05128700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 43199) UID: 1000 2021-02-20T15:48:40.197 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:48:46.997 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T15:48:46.997 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:48:46.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:48:46.998 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T15:48:46.998 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T15:48:46.999 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T15:48:47.001 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T15:48:47.002 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T15:48:49.535 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8904442485469423) > weight (0.25) 2021-02-20T15:48:49.536 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4159683393660203) > weight (0.25) 2021-02-20T15:48:49.536 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6518868216077444) > weight (0.25) 2021-02-20T15:48:49.536 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.70237485714703) > weight (0.25) 2021-02-20T15:48:49.537 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T15:50:37.538 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8243353027184093) > weight (0.25) 2021-02-20T15:50:37.539 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8091060624091562) > weight (0.25) 2021-02-20T15:50:37.540 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.33188794937532906) > weight (0.25) 2021-02-20T15:50:37.540 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T15:50:37.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T15:50:37.542 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:50:37.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T15:50:37.537+0000 7f7b642da700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 42087) UID: 1000 2021-02-20T15:50:37.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:50:40.003 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7715512624401667) > weight (0.25) 2021-02-20T15:50:40.003 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5189078215085768) > weight (0.25) 2021-02-20T15:50:40.004 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.27442081356163195) > weight (0.25) 2021-02-20T15:50:40.004 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3769248934937447) > weight (0.25) 2021-02-20T15:50:40.004 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T15:50:45.543 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T15:50:45.544 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:50:45.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:50:45.545 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T15:50:45.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T15:50:45.547 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T15:50:45.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T15:50:45.551 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T15:51:54.552 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:51:54.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:51:54.554 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T15:51:54.554 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:51:54.554 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:51:54.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T15:51:54.551+0000 7f9af2c47700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 43567) UID: 1000 2021-02-20T15:51:54.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:52:00.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:52:00.657 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:52:00.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:52:00.658 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:52:00.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:52:00.661 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T15:52:39.005 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5877226332847879) > weight (0.25) 2021-02-20T15:52:39.006 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.44093816120507245) > weight (0.25) 2021-02-20T15:52:39.006 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T15:52:39.007 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T15:52:39.007 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T15:52:39.008 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T15:52:39.006+0000 7f4322301700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 43841) UID: 1000 2021-02-20T15:52:39.209 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:52:43.008 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T15:52:43.008 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:52:43.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:52:43.009 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T15:52:43.009 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T15:52:43.010 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T15:52:43.012 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T15:52:43.012 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T15:53:46.662 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.72033145084351) > weight (0.25) 2021-02-20T15:53:46.663 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T15:53:46.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T15:53:46.664 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T15:53:46.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T15:53:46.664+0000 7f3336aaa700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 43750) UID: 1000 2021-02-20T15:53:47.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:53:52.665 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T15:53:52.665 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:53:52.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:53:52.666 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T15:53:52.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T15:53:52.667 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T15:53:52.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T15:53:52.670 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T15:54:15.013 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7072536660767208) > weight (0.25) 2021-02-20T15:54:15.014 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.33192990829196545) > weight (0.25) 2021-02-20T15:54:15.014 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T15:54:15.015 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T15:54:15.015 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T15:54:15.016 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T15:54:15.016+0000 7fbd8d8df700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 44126) UID: 1000 2021-02-20T15:54:15.217 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:54:16.016 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T15:54:16.017 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:54:16.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:54:16.017 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T15:54:16.018 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T15:54:16.018 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T15:54:16.021 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T15:54:16.021 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T15:55:00.671 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:55:00.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:55:00.672 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T15:55:00.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T15:55:00.670+0000 7f0679ec0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 44034) UID: 1000 2021-02-20T15:55:00.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:55:06.673 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:55:06.674 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:55:06.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:55:06.674 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:55:06.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:55:06.675 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:55:06.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:55:06.678 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T15:55:31.022 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.32370119523151386) > weight (0.25) 2021-02-20T15:55:31.023 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5158260461587464) > weight (0.25) 2021-02-20T15:55:31.023 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9137980625957784) > weight (0.25) 2021-02-20T15:55:31.024 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9324747104889405) > weight (0.25) 2021-02-20T15:55:31.024 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T15:55:54.443 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T15:55:54.436+0000 7fb8259e4700 -1 reset not still connected to 0x557840bbe410 2021-02-20T15:55:54.443 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T15:55:54.436+0000 7fb8259e4700 -1 reset not still connected to 0x557855c1d520 2021-02-20T15:55:54.444 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T15:55:54.436+0000 7fb8259e4700 -1 reset not still connected to 0x557855c1d5f0 2021-02-20T15:55:54.444 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T15:55:54.436+0000 7fb8259e4700 -1 reset not still connected to 0x557855c1f0c0 2021-02-20T15:55:54.444 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T15:55:54.436+0000 7fb8259e4700 -1 reset not still connected to 0x55785824a220 2021-02-20T15:56:31.679 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5804727182582854) > weight (0.25) 2021-02-20T15:56:31.680 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T15:56:31.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T15:56:31.681 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T15:56:31.681 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T15:56:31.682 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:56:31.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T15:56:31.680+0000 7f257767b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 44225) UID: 1000 2021-02-20T15:56:31.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:56:37.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:56:37.784 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T15:56:37.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T15:56:37.784 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T15:56:37.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T15:56:37.787 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T15:56:55.024 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6909933724450015) > weight (0.25) 2021-02-20T15:56:55.025 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.25515885440890795) > weight (0.25) 2021-02-20T15:56:55.025 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9339370659006404) > weight (0.25) 2021-02-20T15:56:55.025 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7492006154460029) > weight (0.25) 2021-02-20T15:56:55.025 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T15:57:52.651 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T15:57:52.646+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6cfb65520 2021-02-20T15:57:52.652 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T15:57:52.646+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6e58ad8e0 2021-02-20T15:58:01.788 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.45323324859973746) > weight (0.25) 2021-02-20T15:58:01.788 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9814597412255117) > weight (0.25) 2021-02-20T15:58:01.789 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7003021572271843) > weight (0.25) 2021-02-20T15:58:01.789 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8458171370923764) > weight (0.25) 2021-02-20T15:58:01.789 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T15:58:14.026 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3281855452241943) > weight (0.25) 2021-02-20T15:58:14.027 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3223779140723715) > weight (0.25) 2021-02-20T15:58:14.027 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T15:58:14.028 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T15:58:14.028 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T15:58:14.029 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T15:58:14.025+0000 7faf490ac700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 44316) UID: 1000 2021-02-20T15:58:14.229 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:58:16.029 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T15:58:16.029 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:58:16.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:58:16.030 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T15:58:16.031 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T15:58:16.031 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T15:58:16.034 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T15:58:16.034 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T15:59:27.035 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.33889310345963564) > weight (0.25) 2021-02-20T15:59:27.036 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T15:59:27.036 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T15:59:27.037 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T15:59:27.038 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T15:59:27.035+0000 7fcf1335b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 41718) UID: 1000 2021-02-20T15:59:27.238 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:59:30.038 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T15:59:30.038 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:59:30.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:59:30.039 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T15:59:30.039 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T15:59:30.040 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T15:59:30.043 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T15:59:30.043 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T15:59:47.790 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8970903127427542) > weight (0.25) 2021-02-20T15:59:47.791 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7889867331091949) > weight (0.25) 2021-02-20T15:59:47.791 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6506151594098233) > weight (0.25) 2021-02-20T15:59:47.791 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5284781027851273) > weight (0.25) 2021-02-20T15:59:47.792 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T16:00:54.044 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:00:54.046 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:00:54.046 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T16:00:54.047 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T16:00:54.045+0000 7fdb94e82700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 43476) UID: 1000 2021-02-20T16:00:54.248 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:00:58.047 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:00:58.047 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:00:58.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:00:58.048 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:00:58.048 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:00:58.049 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:00:58.052 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:00:58.052 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T16:01:31.793 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:01:31.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:01:31.794 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T16:01:31.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T16:01:31.792+0000 7f7247e8e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 44410) UID: 1000 2021-02-20T16:01:31.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:01:37.796 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:01:37.796 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:01:37.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:01:37.797 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:01:37.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:01:37.798 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:01:37.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:01:37.801 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T16:01:47.251 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:01:47.247+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6cfb65520 2021-02-20T16:01:47.252 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:01:47.247+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6e58ad8e0 2021-02-20T16:02:20.053 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5899879582751442) > weight (0.25) 2021-02-20T16:02:20.053 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8592900263275799) > weight (0.25) 2021-02-20T16:02:20.054 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7324142979969964) > weight (0.25) 2021-02-20T16:02:20.054 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T16:02:20.055 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T16:02:20.055 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T16:02:20.056 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T16:02:20.054+0000 7fd4c65f2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 43658) UID: 1000 2021-02-20T16:02:20.256 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:02:22.056 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T16:02:22.056 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:02:22.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:02:22.057 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T16:02:22.057 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T16:02:22.058 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T16:02:22.061 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T16:02:22.061 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T16:03:07.802 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7440426245093793) > weight (0.25) 2021-02-20T16:03:07.802 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:03:07.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:03:07.803 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T16:03:07.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T16:03:07.801+0000 7fe5a70da700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 44502) UID: 1000 2021-02-20T16:03:08.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:03:08.805 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:03:08.805 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:03:08.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:03:08.806 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:03:08.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:03:08.806 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:03:08.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:03:08.809 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T16:03:47.062 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:03:47.063 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:03:47.064 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T16:03:47.064 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T16:03:47.064+0000 7efc99604700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 44846) UID: 1000 2021-02-20T16:03:47.265 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:03:49.064 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:03:49.065 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:03:49.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:03:49.066 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:03:49.066 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:03:49.066 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:03:49.069 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:03:49.069 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T16:04:52.811 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:04:52.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:04:52.812 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T16:04:52.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T16:04:52.810+0000 7fb89a429700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 44937) UID: 1000 2021-02-20T16:04:53.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:04:53.813 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:04:53.814 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:04:53.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:04:53.814 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:04:53.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:04:53.815 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:04:53.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:04:53.820 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T16:05:39.071 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9746915338885783) > weight (0.25) 2021-02-20T16:05:39.071 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7081565684878073) > weight (0.25) 2021-02-20T16:05:39.071 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.44268285461982726) > weight (0.25) 2021-02-20T16:05:39.072 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8135580085802113) > weight (0.25) 2021-02-20T16:05:39.072 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T16:06:11.822 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5955968019326877) > weight (0.25) 2021-02-20T16:06:11.873 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:06:11.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:06:11.874 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T16:06:11.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T16:06:11.872+0000 7f98053bf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 45120) UID: 1000 2021-02-20T16:06:12.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:06:13.876 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:06:13.876 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:06:13.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:06:13.877 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:06:13.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:06:13.878 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:06:13.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:06:13.881 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T16:07:08.073 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4848163750347064) > weight (0.25) 2021-02-20T16:07:08.073 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:07:08.074 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:07:08.074 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T16:07:08.075 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T16:07:08.075+0000 7f912e762700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 44686) UID: 1000 2021-02-20T16:07:08.276 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:07:17.075 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:07:17.075 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:07:17.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:07:17.076 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:07:17.076 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:07:17.077 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:07:17.079 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:07:17.080 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T16:08:01.882 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5369266813823608) > weight (0.25) 2021-02-20T16:08:01.882 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9041530898858927) > weight (0.25) 2021-02-20T16:08:01.882 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8271631149288798) > weight (0.25) 2021-02-20T16:08:01.883 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4651288843274849) > weight (0.25) 2021-02-20T16:08:01.883 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T16:08:53.081 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6600742906877843) > weight (0.25) 2021-02-20T16:08:53.082 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6754733868615594) > weight (0.25) 2021-02-20T16:08:53.082 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T16:08:53.083 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T16:08:53.083 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T16:08:53.084 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T16:08:53.080+0000 7f5a077ee700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 44594) UID: 1000 2021-02-20T16:08:53.285 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:08:56.084 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T16:08:56.085 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:08:56.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:08:56.086 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T16:08:56.086 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T16:08:56.086 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T16:08:56.089 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T16:08:56.089 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T16:09:42.884 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:09:42.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:09:42.885 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T16:09:42.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T16:09:42.882+0000 7fad5834c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 45303) UID: 1000 2021-02-20T16:09:43.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:09:48.886 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:09:48.886 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:09:48.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:09:48.887 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:09:48.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:09:48.888 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:09:48.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:09:48.891 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T16:10:19.091 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5378268654633206) > weight (0.25) 2021-02-20T16:10:19.091 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.49793287989838697) > weight (0.25) 2021-02-20T16:10:19.092 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T16:10:19.093 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T16:10:19.093 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T16:10:19.094 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T16:10:19.089+0000 7f3c58d4f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 45582) UID: 1000 2021-02-20T16:10:19.294 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:10:23.094 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T16:10:23.094 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:10:23.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:10:23.095 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T16:10:23.095 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T16:10:23.096 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T16:10:23.098 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T16:10:23.099 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T16:11:16.892 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5077669634012215) > weight (0.25) 2021-02-20T16:11:16.892 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3409259971854939) > weight (0.25) 2021-02-20T16:11:16.892 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6533626103392137) > weight (0.25) 2021-02-20T16:11:16.893 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6767206877667488) > weight (0.25) 2021-02-20T16:11:16.893 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T16:12:14.185 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:12:14.178+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a47512b0 2021-02-20T16:12:14.194 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:12:14.186+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a56775f0 2021-02-20T16:12:14.194 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:12:14.186+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a600f260 2021-02-20T16:12:14.194 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:12:14.186+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b6c86b60 2021-02-20T16:12:14.195 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:12:14.186+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6e58ab110 2021-02-20T16:12:14.195 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:12:14.186+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6e58aba00 2021-02-20T16:12:16.099 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2854147319010567) > weight (0.25) 2021-02-20T16:12:16.100 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2636008316033652) > weight (0.25) 2021-02-20T16:12:16.100 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T16:12:16.101 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T16:12:16.101 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T16:12:16.102 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T16:12:16.098+0000 7f0a55384700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 45766) UID: 1000 2021-02-20T16:12:16.302 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:12:23.103 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T16:12:23.103 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:12:23.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:12:23.104 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T16:12:23.105 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T16:12:23.105 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T16:12:23.108 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T16:12:23.109 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T16:12:57.894 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9903526569914847) > weight (0.25) 2021-02-20T16:12:57.895 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8867584672137938) > weight (0.25) 2021-02-20T16:12:57.895 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.27740489924173684) > weight (0.25) 2021-02-20T16:12:57.896 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T16:12:57.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T16:12:57.897 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T16:12:57.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T16:12:57.893+0000 7fd01a7ae700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 43942) UID: 1000 2021-02-20T16:12:58.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:13:06.898 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T16:13:06.899 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:13:06.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:13:06.899 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T16:13:06.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T16:13:06.900 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T16:13:06.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T16:13:06.903 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T16:13:34.110 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:13:34.111 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:13:34.112 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T16:13:34.113 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T16:13:34.108+0000 7f64edef9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 45211) UID: 1000 2021-02-20T16:13:34.313 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:13:38.113 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:13:38.113 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:13:38.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:13:38.114 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:13:38.114 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:13:38.115 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:13:38.118 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:13:38.118 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T16:14:31.903 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:14:31.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:14:31.905 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:14:31.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T16:14:31.903+0000 7fc475770700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 45674) UID: 1000 2021-02-20T16:14:32.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:14:36.906 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:14:36.907 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:14:36.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:14:36.907 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:14:36.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:14:36.908 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:14:36.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:14:36.911 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T16:15:03.119 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9845438293215959) > weight (0.25) 2021-02-20T16:15:03.120 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7680227662910275) > weight (0.25) 2021-02-20T16:15:03.120 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T16:15:03.121 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T16:15:03.121 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T16:15:03.122 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T16:15:03.118+0000 7f07c871d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 45858) UID: 1000 2021-02-20T16:15:03.322 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:15:05.122 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T16:15:05.123 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:15:05.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:15:05.124 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T16:15:05.124 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T16:15:05.124 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T16:15:05.127 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T16:15:05.127 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T16:15:42.404 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T16:15:42.400+0000 7fb8259e4700 -1 reset not still connected to 0x557840bc0e50 2021-02-20T16:15:42.405 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T16:15:42.400+0000 7fb8259e4700 -1 reset not still connected to 0x557855c1d520 2021-02-20T16:15:42.405 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T16:15:42.400+0000 7fb8259e4700 -1 reset not still connected to 0x557855c1d5f0 2021-02-20T16:15:42.405 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T16:15:42.400+0000 7fb8259e4700 -1 reset not still connected to 0x557855c1f0c0 2021-02-20T16:15:42.405 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T16:15:42.400+0000 7fb8259e4700 -1 reset not still connected to 0x55785824a220 2021-02-20T16:15:42.406 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T16:15:42.400+0000 7fb8259e4700 -1 reset not still connected to 0x55785824ab10 2021-02-20T16:15:42.406 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T16:15:42.400+0000 7fb8259e4700 -1 reset not still connected to 0x55785824b260 2021-02-20T16:15:42.406 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T16:15:42.400+0000 7fb8259e4700 -1 reset not still connected to 0x55785824bb50 2021-02-20T16:15:42.406 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T16:15:42.400+0000 7fb8259e4700 -1 reset not still connected to 0x55785824bcf0 2021-02-20T16:15:42.407 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T16:15:42.400+0000 7fb8259e4700 -1 reset not still connected to 0x55785c64e000 2021-02-20T16:15:42.407 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T16:15:42.400+0000 7fb8259e4700 -1 reset not still connected to 0x55785c64f110 2021-02-20T16:15:42.407 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T16:15:42.400+0000 7fb8259e4700 -1 reset not still connected to 0x55785c651b50 2021-02-20T16:15:42.407 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T16:15:42.400+0000 7fb8259e4700 -1 reset not still connected to 0x55785c651e90 2021-02-20T16:16:12.912 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4558767997915707) > weight (0.25) 2021-02-20T16:16:12.912 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:16:12.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:16:12.913 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T16:16:12.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T16:16:12.912+0000 7f460b176700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 45398) UID: 1000 2021-02-20T16:16:13.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:16:13.914 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:16:13.915 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:16:13.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:16:13.915 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:16:13.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:16:13.916 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:16:13.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:16:13.919 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T16:16:43.128 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5519690624081517) > weight (0.25) 2021-02-20T16:16:43.128 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:16:43.129 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:16:43.129 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T16:16:43.130 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T16:16:43.127+0000 7f78001d8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 45489) UID: 1000 2021-02-20T16:16:43.330 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:16:52.130 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:16:52.131 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:16:52.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:16:52.132 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:16:52.132 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:16:52.133 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:16:52.137 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:16:52.137 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T16:17:56.139 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.36609118893654413) > weight (0.25) 2021-02-20T16:17:56.140 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:17:56.140 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:17:56.141 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T16:17:56.142 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T16:17:56.137+0000 7f2a0a8a9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 46409) UID: 1000 2021-02-20T16:17:56.342 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:17:56.920 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3253561317304481) > weight (0.25) 2021-02-20T16:17:56.921 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5962832903793513) > weight (0.25) 2021-02-20T16:17:56.921 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8923066495639351) > weight (0.25) 2021-02-20T16:17:56.922 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.418674041242591) > weight (0.25) 2021-02-20T16:17:56.922 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T16:18:05.141 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:18:05.142 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:18:05.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:18:05.143 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:18:05.143 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:18:05.143 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:18:05.146 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:18:05.146 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T16:19:33.924 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9643809820217852) > weight (0.25) 2021-02-20T16:19:33.925 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2617627934165283) > weight (0.25) 2021-02-20T16:19:33.925 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3979227443594272) > weight (0.25) 2021-02-20T16:19:33.925 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9880589240964867) > weight (0.25) 2021-02-20T16:19:33.925 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T16:19:39.147 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:19:39.148 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:19:39.148 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:19:39.149 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T16:19:39.146+0000 7f9e84a48700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 46041) UID: 1000 2021-02-20T16:19:39.350 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:19:44.149 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:19:44.150 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:19:44.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:19:44.151 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:19:44.151 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:19:44.151 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:19:44.155 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:19:44.155 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T16:20:22.334 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:20:22.325+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6c083db50 2021-02-20T16:20:22.335 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:20:22.329+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ca080b10 2021-02-20T16:21:08.156 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6111815434114519) > weight (0.25) 2021-02-20T16:21:08.157 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:21:08.158 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:21:08.158 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:21:08.159 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T16:21:08.156+0000 7f0996a6c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 46505) UID: 1000 2021-02-20T16:21:08.359 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:21:11.926 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.925516654750082) > weight (0.25) 2021-02-20T16:21:11.926 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:21:11.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:21:11.927 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T16:21:11.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T16:21:11.924+0000 7f4bb075e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 46318) UID: 1000 2021-02-20T16:21:12.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:21:13.159 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:21:13.160 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:21:13.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:21:13.161 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:21:13.161 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:21:13.161 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:21:13.164 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:21:13.164 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T16:21:15.929 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:21:15.929 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:21:15.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:21:15.930 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:21:15.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:21:15.930 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:21:15.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:21:15.933 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T16:22:43.935 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:22:43.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:22:43.937 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:22:43.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T16:22:43.934+0000 7f2a29126700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 46133) UID: 1000 2021-02-20T16:22:44.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:22:48.938 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:22:48.938 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:22:48.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:22:48.939 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:22:48.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:22:48.940 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:22:48.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:22:48.943 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T16:23:11.165 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7029279031651658) > weight (0.25) 2021-02-20T16:23:11.166 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6438864956624256) > weight (0.25) 2021-02-20T16:23:11.166 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.385191573595143) > weight (0.25) 2021-02-20T16:23:11.166 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7673501260490527) > weight (0.25) 2021-02-20T16:23:11.167 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T16:24:38.168 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6429729005437174) > weight (0.25) 2021-02-20T16:24:38.168 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5260872471782351) > weight (0.25) 2021-02-20T16:24:38.168 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T16:24:38.169 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T16:24:38.170 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T16:24:38.171 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T16:24:38.166+0000 7f1a5968a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 46227) UID: 1000 2021-02-20T16:24:38.371 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:24:41.171 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T16:24:41.172 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:24:41.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:24:41.173 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T16:24:41.173 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T16:24:41.173 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T16:24:41.176 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T16:24:41.176 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T16:24:44.944 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6560827397371841) > weight (0.25) 2021-02-20T16:24:44.945 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:24:44.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:24:44.946 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T16:24:44.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T16:24:44.942+0000 7f8d6e931700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 46782) UID: 1000 2021-02-20T16:24:45.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:24:53.947 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:24:53.947 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:24:53.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:24:53.948 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:24:53.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:24:53.948 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:24:53.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:24:53.951 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T16:26:05.178 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:26:05.180 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:26:05.180 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T16:26:05.181 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T16:26:05.176+0000 7f8bf16e6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 46596) UID: 1000 2021-02-20T16:26:05.381 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:26:09.182 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:26:09.182 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:26:09.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:26:09.183 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:26:09.183 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:26:09.184 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:26:09.187 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:26:09.187 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T16:26:34.952 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.33967961265213764) > weight (0.25) 2021-02-20T16:26:34.953 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.43977350273141924) > weight (0.25) 2021-02-20T16:26:34.953 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3042786873318821) > weight (0.25) 2021-02-20T16:26:34.953 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3475587006784494) > weight (0.25) 2021-02-20T16:26:34.953 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T16:27:43.188 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9423886500114762) > weight (0.25) 2021-02-20T16:27:43.189 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.35267434968836275) > weight (0.25) 2021-02-20T16:27:43.189 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8451922385425014) > weight (0.25) 2021-02-20T16:27:43.189 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8346403870648683) > weight (0.25) 2021-02-20T16:27:43.189 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T16:27:55.952 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T16:27:55.945+0000 7f4a1bb72700 -1 reset not still connected to 0x55fc710a9c20 2021-02-20T16:27:55.953 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T16:27:55.949+0000 7f4a1bb72700 -1 reset not still connected to 0x55fc8ac5d190 2021-02-20T16:27:55.965 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T16:27:55.949+0000 7f4a1bb72700 -1 reset not still connected to 0x55fcbe7402f0 2021-02-20T16:28:09.954 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:28:09.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:28:09.956 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T16:28:09.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T16:28:09.949+0000 7f2ad51af700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 46875) UID: 1000 2021-02-20T16:28:10.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:28:17.956 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:28:17.957 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:28:17.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:28:17.957 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:28:17.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:28:17.958 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:28:17.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:28:17.961 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T16:28:51.191 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3348035727351738) > weight (0.25) 2021-02-20T16:28:51.192 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6116255672477523) > weight (0.25) 2021-02-20T16:28:51.192 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T16:28:51.193 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T16:28:51.195 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T16:28:51.196 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T16:28:51.192+0000 7f2e3a366700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 46967) UID: 1000 2021-02-20T16:28:51.397 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:29:00.196 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T16:29:00.196 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:29:00.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:29:00.197 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T16:29:00.198 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T16:29:00.198 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T16:29:00.202 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T16:29:00.203 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T16:29:52.962 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6671389643982559) > weight (0.25) 2021-02-20T16:29:52.962 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.45346947364519885) > weight (0.25) 2021-02-20T16:29:52.963 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T16:29:52.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T16:29:52.963 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:29:52.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T16:29:52.958+0000 7f28ca18a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 40494) UID: 1000 2021-02-20T16:29:53.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:29:57.965 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T16:29:57.965 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:29:57.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:29:57.966 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T16:29:57.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T16:29:57.967 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T16:29:57.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T16:29:57.970 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T16:30:57.204 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9403998472155004) > weight (0.25) 2021-02-20T16:30:57.205 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8644669184329707) > weight (0.25) 2021-02-20T16:30:57.205 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6332504065645489) > weight (0.25) 2021-02-20T16:30:57.205 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T16:30:57.206 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T16:30:57.206 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T16:30:57.207 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T16:30:57.204+0000 7fec4f550700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 45028) UID: 1000 2021-02-20T16:30:57.407 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:31:05.207 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T16:31:05.208 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:31:05.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:31:05.208 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T16:31:05.209 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T16:31:05.209 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T16:31:05.212 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T16:31:05.212 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T16:31:16.971 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5157770482676267) > weight (0.25) 2021-02-20T16:31:16.981 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.48884101988608963) > weight (0.25) 2021-02-20T16:31:16.981 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6956091145125639) > weight (0.25) 2021-02-20T16:31:16.982 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4320811949992701) > weight (0.25) 2021-02-20T16:31:16.982 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T16:31:30.450 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:31:30.447+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a391a750 2021-02-20T16:31:30.451 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:31:30.447+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a600c340 2021-02-20T16:31:30.451 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:31:30.447+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6c083c080 2021-02-20T16:31:30.453 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:31:30.447+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ca0810c0 2021-02-20T16:31:30.453 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:31:30.447+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ca081330 2021-02-20T16:33:00.213 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7331247668084699) > weight (0.25) 2021-02-20T16:33:00.221 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7220885364833486) > weight (0.25) 2021-02-20T16:33:00.221 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4596621682686467) > weight (0.25) 2021-02-20T16:33:00.221 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T16:33:00.222 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T16:33:00.222 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T16:33:00.223 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T16:33:00.221+0000 7f5ec4929700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 47528) UID: 1000 2021-02-20T16:33:00.423 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:33:09.223 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T16:33:09.223 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:33:09.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:33:09.224 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T16:33:09.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T16:33:09.224 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T16:33:09.227 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T16:33:09.227 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T16:33:13.983 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9308655971530239) > weight (0.25) 2021-02-20T16:33:13.984 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5457997810712053) > weight (0.25) 2021-02-20T16:33:13.984 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T16:33:13.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T16:33:13.985 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T16:33:13.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T16:33:13.977+0000 7f31b4d8f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 47436) UID: 1000 2021-02-20T16:33:14.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:33:19.986 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T16:33:19.986 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:33:19.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:33:19.987 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T16:33:19.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T16:33:19.988 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T16:33:19.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T16:33:19.991 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T16:34:24.228 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6766896202212589) > weight (0.25) 2021-02-20T16:34:24.236 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5315788626689977) > weight (0.25) 2021-02-20T16:34:24.236 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T16:34:24.237 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T16:34:24.237 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T16:34:24.238 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T16:34:24.238 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:34:24.239 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T16:34:24.235+0000 7f8c5fa1f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 47337) UID: 1000 2021-02-20T16:34:24.438 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:34:30.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:34:30.349 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T16:34:30.349 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T16:34:30.349 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T16:34:30.352 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T16:34:30.353 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T16:34:43.992 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.403739663282237) > weight (0.25) 2021-02-20T16:34:44.000 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4166397234494035) > weight (0.25) 2021-02-20T16:34:44.000 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T16:34:44.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T16:34:44.001 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T16:34:44.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T16:34:43.998+0000 7fde87553700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 47711) UID: 1000 2021-02-20T16:34:44.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:34:47.002 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T16:34:47.002 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:34:47.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:34:47.003 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T16:34:47.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T16:34:47.004 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T16:34:47.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T16:34:47.007 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T16:35:09.491 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:35:09.486+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a6d6e3c0 2021-02-20T16:35:09.491 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:35:09.486+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b6c895a0 2021-02-20T16:35:09.492 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:35:09.486+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ca07fad0 2021-02-20T16:35:09.492 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:35:09.486+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6cfb663c0 2021-02-20T16:36:17.008 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:36:17.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:36:17.016 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T16:36:17.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T16:36:17.012+0000 7fa7457a2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 47246) UID: 1000 2021-02-20T16:36:17.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:36:24.017 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:36:24.017 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:36:24.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:36:24.018 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:36:24.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:36:24.019 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:36:24.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:36:24.022 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T16:36:27.354 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6081333312982528) > weight (0.25) 2021-02-20T16:36:27.354 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.28597801201339845) > weight (0.25) 2021-02-20T16:36:27.355 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6903766097032531) > weight (0.25) 2021-02-20T16:36:27.355 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3093655905027505) > weight (0.25) 2021-02-20T16:36:27.355 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T16:37:53.356 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9055374856762894) > weight (0.25) 2021-02-20T16:37:53.367 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9084791682431256) > weight (0.25) 2021-02-20T16:37:53.367 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5995871670493726) > weight (0.25) 2021-02-20T16:37:53.367 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6847686227384361) > weight (0.25) 2021-02-20T16:37:53.368 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T16:38:18.023 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5658623130884215) > weight (0.25) 2021-02-20T16:38:18.031 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:38:18.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:38:18.033 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T16:38:18.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T16:38:18.033+0000 7fb58cb92700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 47058) UID: 1000 2021-02-20T16:38:18.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:38:22.034 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:38:22.034 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:38:22.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:38:22.036 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:38:22.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:38:22.036 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:38:22.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:38:22.040 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T16:38:57.369 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5805474889339368) > weight (0.25) 2021-02-20T16:38:57.384 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:38:57.385 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:38:57.385 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T16:38:57.386 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T16:38:57.384+0000 7f0ee99f3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 46691) UID: 1000 2021-02-20T16:38:57.586 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:39:05.386 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:39:05.386 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:39:05.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:39:05.387 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:39:05.387 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:39:05.388 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:39:05.390 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:39:05.391 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T16:39:27.041 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.32133678610511107) > weight (0.25) 2021-02-20T16:39:27.045 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3779296564421518) > weight (0.25) 2021-02-20T16:39:27.045 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6555990675457267) > weight (0.25) 2021-02-20T16:39:27.046 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.759127516248731) > weight (0.25) 2021-02-20T16:39:27.047 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T16:39:39.610 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T16:39:39.606+0000 7fb8259e4700 -1 reset not still connected to 0x557855c1c340 2021-02-20T16:40:41.391 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3252881804280433) > weight (0.25) 2021-02-20T16:40:41.407 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:40:41.408 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:40:41.410 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T16:40:41.411 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T16:40:41.405+0000 7f671b6b6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 48180) UID: 1000 2021-02-20T16:40:41.610 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:40:44.411 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:40:44.411 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:40:44.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:40:44.412 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:40:44.412 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:40:44.413 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:40:44.415 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:40:44.416 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T16:41:26.048 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:41:26.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:41:26.056 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T16:41:26.057 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:41:26.057 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:41:26.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T16:41:26.056+0000 7fed46dda700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 47988) UID: 1000 2021-02-20T16:41:26.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:41:32.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:41:32.159 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:41:32.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:41:32.160 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:41:32.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:41:32.163 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T16:42:35.416 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4079576503883765) > weight (0.25) 2021-02-20T16:42:35.429 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9461875177920017) > weight (0.25) 2021-02-20T16:42:35.429 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T16:42:35.430 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T16:42:35.430 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T16:42:35.431 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T16:42:35.426+0000 7f18324e5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 47803) UID: 1000 2021-02-20T16:42:35.632 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:42:41.431 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T16:42:41.432 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:42:41.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:42:41.433 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T16:42:41.433 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T16:42:41.433 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T16:42:41.437 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T16:42:41.437 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T16:42:43.164 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.38057044136505214) > weight (0.25) 2021-02-20T16:42:43.164 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.47103445962443924) > weight (0.25) 2021-02-20T16:42:43.165 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T16:42:43.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T16:42:43.165 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:42:43.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T16:42:43.166+0000 7f6dd58f3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 47894) UID: 1000 2021-02-20T16:42:43.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:42:48.166 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T16:42:48.167 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:42:48.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:42:48.168 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T16:42:48.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T16:42:48.168 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T16:42:48.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T16:42:48.171 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T16:44:13.438 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:44:13.440 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:44:13.440 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T16:44:13.441 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T16:44:13.439+0000 7f8239b30700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 47153) UID: 1000 2021-02-20T16:44:13.641 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:44:18.172 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5455619447522746) > weight (0.25) 2021-02-20T16:44:18.173 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5843278528631757) > weight (0.25) 2021-02-20T16:44:18.174 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8343781696921574) > weight (0.25) 2021-02-20T16:44:18.174 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.28297969449937677) > weight (0.25) 2021-02-20T16:44:18.174 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T16:44:22.441 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:44:22.441 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:44:22.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:44:22.443 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:44:22.443 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:44:22.444 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:44:22.447 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:44:22.448 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T16:45:25.449 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.47268670428092574) > weight (0.25) 2021-02-20T16:45:25.450 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.25643761348330985) > weight (0.25) 2021-02-20T16:45:25.450 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3848045570508989) > weight (0.25) 2021-02-20T16:45:25.450 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5878828519066749) > weight (0.25) 2021-02-20T16:45:25.450 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T16:45:33.176 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.27855680502798696) > weight (0.25) 2021-02-20T16:45:33.177 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:45:33.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:45:33.178 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T16:45:33.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T16:45:33.177+0000 7f2749779700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 48080) UID: 1000 2021-02-20T16:45:33.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:45:35.179 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:45:35.179 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:45:35.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:45:35.180 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:45:35.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:45:35.181 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:45:35.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:45:35.183 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T16:46:13.608 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T16:46:13.600+0000 7f4a1bb72700 -1 reset not still connected to 0x55fcabadaf20 2021-02-20T16:46:59.185 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7450030695434708) > weight (0.25) 2021-02-20T16:46:59.186 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5790250325993564) > weight (0.25) 2021-02-20T16:46:59.186 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9116825365225671) > weight (0.25) 2021-02-20T16:46:59.186 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6447556890497292) > weight (0.25) 2021-02-20T16:46:59.186 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T16:47:07.451 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8909555285062045) > weight (0.25) 2021-02-20T16:47:07.452 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.32938708236114744) > weight (0.25) 2021-02-20T16:47:07.452 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T16:47:07.453 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T16:47:07.454 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T16:47:07.455 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T16:47:07.450+0000 7fc92af36700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 48456) UID: 1000 2021-02-20T16:47:07.655 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:47:14.454 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T16:47:14.455 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:47:14.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:47:14.456 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T16:47:14.456 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T16:47:14.456 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T16:47:14.459 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T16:47:14.459 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T16:48:16.188 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:48:16.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:48:16.190 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T16:48:16.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T16:48:16.189+0000 7fef368a2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 48365) UID: 1000 2021-02-20T16:48:16.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:48:17.460 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.34324802207011573) > weight (0.25) 2021-02-20T16:48:17.461 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:48:17.461 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:48:17.461 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T16:48:17.462 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T16:48:17.461+0000 7efccb9c9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 48274) UID: 1000 2021-02-20T16:48:17.663 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:48:19.191 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:48:19.192 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:48:19.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:48:19.192 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:48:19.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:48:19.193 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:48:19.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:48:19.196 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T16:48:23.463 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:48:23.463 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:48:23.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:48:23.464 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:48:23.464 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:48:23.464 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:48:23.467 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:48:23.467 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T16:50:02.198 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9073440439846028) > weight (0.25) 2021-02-20T16:50:02.199 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3621146545010594) > weight (0.25) 2021-02-20T16:50:02.199 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T16:50:02.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T16:50:02.201 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T16:50:02.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T16:50:02.198+0000 7f4176fb7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 48547) UID: 1000 2021-02-20T16:50:02.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:50:03.201 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T16:50:03.202 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:50:03.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:50:03.202 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T16:50:03.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T16:50:03.203 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T16:50:03.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T16:50:03.206 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T16:50:05.469 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7632821563207043) > weight (0.25) 2021-02-20T16:50:05.469 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6454391976320667) > weight (0.25) 2021-02-20T16:50:05.469 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T16:50:05.470 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T16:50:05.470 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T16:50:05.471 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T16:50:05.470+0000 7faa571c2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 48825) UID: 1000 2021-02-20T16:50:05.671 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:50:11.471 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T16:50:11.471 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:50:11.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:50:11.472 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T16:50:11.472 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T16:50:11.473 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T16:50:11.475 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T16:50:11.476 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T16:51:14.477 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:51:14.478 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:51:14.478 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T16:51:14.479 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:51:14.479 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:51:14.480 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T16:51:14.476+0000 7f59d4ec4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 48639) UID: 1000 2021-02-20T16:51:14.680 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:51:20.207 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:51:20.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:51:20.208 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T16:51:20.208 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:51:20.209 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:51:20.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T16:51:20.208+0000 7fce68e65700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 48916) UID: 1000 2021-02-20T16:51:20.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:51:20.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:51:20.581 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:51:20.581 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:51:20.582 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:51:20.584 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:51:20.585 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T16:51:26.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:51:26.311 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:51:26.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:51:26.311 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:51:26.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:51:26.314 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T16:52:30.585 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7083167828675752) > weight (0.25) 2021-02-20T16:52:30.586 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8363159115596082) > weight (0.25) 2021-02-20T16:52:30.586 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9910605741058703) > weight (0.25) 2021-02-20T16:52:30.587 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.25265102442236576) > weight (0.25) 2021-02-20T16:52:30.587 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T16:52:49.315 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8730264393824365) > weight (0.25) 2021-02-20T16:52:49.316 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:52:49.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:52:49.317 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T16:52:49.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T16:52:49.313+0000 7fce236cb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 48734) UID: 1000 2021-02-20T16:52:49.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:52:57.318 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:52:57.319 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:52:57.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:52:57.319 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:52:57.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:52:57.320 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:52:57.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:52:57.323 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T16:53:09.503 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:53:09.497+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b6c895a0 2021-02-20T16:53:09.504 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:53:09.497+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ca07fad0 2021-02-20T16:53:09.504 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:53:09.497+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6cfb663c0 2021-02-20T16:53:54.588 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8838381461204693) > weight (0.25) 2021-02-20T16:53:54.589 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.720874333435169) > weight (0.25) 2021-02-20T16:53:54.589 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5044644897777917) > weight (0.25) 2021-02-20T16:53:54.589 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7493636481654099) > weight (0.25) 2021-02-20T16:53:54.590 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T16:54:30.325 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9057193140884587) > weight (0.25) 2021-02-20T16:54:30.325 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5801896015489538) > weight (0.25) 2021-02-20T16:54:30.325 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T16:54:30.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T16:54:30.327 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T16:54:30.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T16:54:30.323+0000 7f1215b0d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 49100) UID: 1000 2021-02-20T16:54:30.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:54:32.328 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T16:54:32.328 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:54:32.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:54:32.329 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T16:54:32.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T16:54:32.330 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T16:54:32.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T16:54:32.333 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T16:55:16.590 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:55:16.592 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:55:16.593 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T16:55:16.594 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T16:55:16.590+0000 7f4d5af6c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 49283) UID: 1000 2021-02-20T16:55:16.794 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:55:17.594 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:55:17.595 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:55:17.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:55:17.595 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:55:17.596 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:55:17.596 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:55:17.599 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:55:17.599 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T16:56:20.334 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9298700390276396) > weight (0.25) 2021-02-20T16:56:20.335 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8459185140403913) > weight (0.25) 2021-02-20T16:56:20.335 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6100938064246181) > weight (0.25) 2021-02-20T16:56:20.336 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7948122396395838) > weight (0.25) 2021-02-20T16:56:20.336 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T16:56:34.082 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:56:34.079+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ca080b10 2021-02-20T16:56:34.083 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T16:56:34.079+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ca0810c0 2021-02-20T16:56:59.600 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7215405528911595) > weight (0.25) 2021-02-20T16:56:59.601 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.38742098916400847) > weight (0.25) 2021-02-20T16:56:59.601 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8989691868586946) > weight (0.25) 2021-02-20T16:56:59.601 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8864716580961912) > weight (0.25) 2021-02-20T16:56:59.601 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T16:57:30.337 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.43929615562377344) > weight (0.25) 2021-02-20T16:57:30.338 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.42434587511269795) > weight (0.25) 2021-02-20T16:57:30.338 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3237089514857656) > weight (0.25) 2021-02-20T16:57:30.338 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.45459061776189646) > weight (0.25) 2021-02-20T16:57:30.338 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T16:58:56.603 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5404307862740517) > weight (0.25) 2021-02-20T16:58:56.604 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:58:56.605 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:58:56.605 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T16:58:56.606 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:58:56.606 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:58:56.607 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T16:58:56.604+0000 7fb3a3a09700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 49007) UID: 1000 2021-02-20T16:58:56.807 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:59:02.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:59:02.709 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:59:02.709 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:59:02.710 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:59:02.712 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:59:02.713 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T16:59:13.340 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6627851430423598) > weight (0.25) 2021-02-20T16:59:13.340 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:59:13.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:59:13.341 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T16:59:13.341 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:59:13.342 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:59:13.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T16:59:13.339+0000 7f179ce4a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 49466) UID: 1000 2021-02-20T16:59:13.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:59:19.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:59:19.444 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:59:19.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:59:19.444 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:59:19.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:59:19.447 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T17:00:21.449 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.31682987821821773) > weight (0.25) 2021-02-20T17:00:21.449 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7128262985114395) > weight (0.25) 2021-02-20T17:00:21.450 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5705985064011746) > weight (0.25) 2021-02-20T17:00:21.450 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.41652273103070225) > weight (0.25) 2021-02-20T17:00:21.450 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T17:00:29.714 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7714422238488009) > weight (0.25) 2021-02-20T17:00:29.715 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.373469106187287) > weight (0.25) 2021-02-20T17:00:29.715 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8713982163520786) > weight (0.25) 2021-02-20T17:00:29.716 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7910797358513558) > weight (0.25) 2021-02-20T17:00:29.716 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T17:01:45.718 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5164099429931077) > weight (0.25) 2021-02-20T17:01:45.719 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3897226938509656) > weight (0.25) 2021-02-20T17:01:45.719 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2512321368240008) > weight (0.25) 2021-02-20T17:01:45.719 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4949719037300542) > weight (0.25) 2021-02-20T17:01:45.720 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T17:02:13.451 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5145905448272253) > weight (0.25) 2021-02-20T17:02:13.452 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.43668570779265914) > weight (0.25) 2021-02-20T17:02:13.452 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7271793555885806) > weight (0.25) 2021-02-20T17:02:13.452 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T17:02:13.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T17:02:13.454 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T17:02:13.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T17:02:13.454+0000 7fa2a0495700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 45950) UID: 1000 2021-02-20T17:02:13.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:02:17.919 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T17:02:17.914+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b3833d40 2021-02-20T17:02:17.921 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T17:02:17.918+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b8a8f1e0 2021-02-20T17:02:17.921 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T17:02:17.918+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6e58abc70 2021-02-20T17:02:20.455 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T17:02:20.455 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:02:20.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:02:20.456 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T17:02:20.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T17:02:20.457 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T17:02:20.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T17:02:20.460 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T17:03:19.721 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2594025314058622) > weight (0.25) 2021-02-20T17:03:19.722 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T17:03:19.723 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T17:03:19.723 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T17:03:19.724 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T17:03:19.721+0000 7fbe75703700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 49746) UID: 1000 2021-02-20T17:03:19.924 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:03:20.724 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T17:03:20.725 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:03:20.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:03:20.725 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T17:03:20.726 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T17:03:20.726 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T17:03:20.729 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T17:03:20.730 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T17:04:11.462 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7242231865671979) > weight (0.25) 2021-02-20T17:04:11.462 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5016140734389141) > weight (0.25) 2021-02-20T17:04:11.463 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.550456661196654) > weight (0.25) 2021-02-20T17:04:11.463 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4966441279274024) > weight (0.25) 2021-02-20T17:04:11.464 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T17:04:24.730 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8076311476192416) > weight (0.25) 2021-02-20T17:04:24.731 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.38702925830709367) > weight (0.25) 2021-02-20T17:04:24.731 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.641216310834178) > weight (0.25) 2021-02-20T17:04:24.732 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9086673015040005) > weight (0.25) 2021-02-20T17:04:24.732 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T17:04:37.941 INFO:tasks.ceph.cluster1.osd.1.smithi170.stderr:2021-02-20T17:04:37.937+0000 7fb102cd5700 -1 reset not still connected to 0x559d40db02f0 2021-02-20T17:04:37.941 INFO:tasks.ceph.cluster1.osd.1.smithi170.stderr:2021-02-20T17:04:37.937+0000 7fb102cd5700 -1 reset not still connected to 0x559d5037a2f0 2021-02-20T17:04:37.941 INFO:tasks.ceph.cluster1.osd.1.smithi170.stderr:2021-02-20T17:04:37.937+0000 7fb102cd5700 -1 reset not still connected to 0x559d5301ef70 2021-02-20T17:04:37.942 INFO:tasks.ceph.cluster1.osd.1.smithi170.stderr:2021-02-20T17:04:37.937+0000 7fb102cd5700 -1 reset not still connected to 0x559d5301fa00 2021-02-20T17:04:37.942 INFO:tasks.ceph.cluster1.osd.1.smithi170.stderr:2021-02-20T17:04:37.937+0000 7fb102cd5700 -1 reset not still connected to 0x559d53020ff0 2021-02-20T17:04:37.942 INFO:tasks.ceph.cluster1.osd.1.smithi170.stderr:2021-02-20T17:04:37.937+0000 7fb102cd5700 -1 reset not still connected to 0x559d53021330 2021-02-20T17:04:37.942 INFO:tasks.ceph.cluster1.osd.1.smithi170.stderr:2021-02-20T17:04:37.937+0000 7fb102cd5700 -1 reset not still connected to 0x559d53021dc0 2021-02-20T17:04:37.943 INFO:tasks.ceph.cluster1.osd.1.smithi170.stderr:2021-02-20T17:04:37.937+0000 7fb102cd5700 -1 reset not still connected to 0x559d53021e90 2021-02-20T17:05:26.733 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4667484926834472) > weight (0.25) 2021-02-20T17:05:26.733 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7107815917532976) > weight (0.25) 2021-02-20T17:05:26.734 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2714274980150617) > weight (0.25) 2021-02-20T17:05:26.734 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2990866904137047) > weight (0.25) 2021-02-20T17:05:26.734 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T17:05:31.465 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:05:31.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:05:31.466 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T17:05:31.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:05:31.465+0000 7fcac9eb5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 49374) UID: 1000 2021-02-20T17:05:31.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:05:37.468 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:05:37.468 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:05:37.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:05:37.470 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:05:37.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:05:37.471 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:05:37.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:05:37.475 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T17:06:46.735 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:06:46.736 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:06:46.737 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T17:06:46.738 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T17:06:46.735+0000 7fe2b26e9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 49653) UID: 1000 2021-02-20T17:06:46.938 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:06:53.737 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:06:53.738 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:06:53.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:06:53.739 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:06:53.739 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:06:53.739 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:06:53.742 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:06:53.742 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T17:07:32.476 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:07:32.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:07:32.477 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T17:07:32.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:07:32.478+0000 7f679335f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 50125) UID: 1000 2021-02-20T17:07:32.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:07:33.478 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:07:33.479 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:07:33.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:07:33.480 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:07:33.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:07:33.480 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:07:33.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:07:33.483 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T17:08:10.744 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9163901927117301) > weight (0.25) 2021-02-20T17:08:10.744 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T17:08:10.745 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T17:08:10.746 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T17:08:10.746 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T17:08:10.741+0000 7f6c75bb1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 50030) UID: 1000 2021-02-20T17:08:10.947 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:08:11.747 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T17:08:11.747 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:08:11.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:08:11.748 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T17:08:11.748 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T17:08:11.749 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T17:08:11.751 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T17:08:11.752 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T17:09:07.485 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:09:07.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:09:07.486 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T17:09:07.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:09:07.484+0000 7f0c60c93700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 50309) UID: 1000 2021-02-20T17:09:07.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:09:13.487 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:09:13.488 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:09:13.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:09:13.489 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:09:13.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:09:13.489 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:09:13.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:09:13.492 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T17:09:38.753 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4529186445209561) > weight (0.25) 2021-02-20T17:09:38.754 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.31450529297071805) > weight (0.25) 2021-02-20T17:09:38.754 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T17:09:38.755 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T17:09:38.755 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T17:09:38.755 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T17:09:38.755 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:09:38.756 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T17:09:38.751+0000 7f3991f06700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 49192) UID: 1000 2021-02-20T17:09:38.956 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:09:44.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:09:44.857 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T17:09:44.857 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T17:09:44.858 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T17:09:44.860 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T17:09:44.861 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T17:10:49.862 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:10:49.863 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:10:49.863 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T17:10:49.864 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T17:10:49.861+0000 7f8edffe8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 50218) UID: 1000 2021-02-20T17:10:50.064 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:10:54.864 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:10:54.864 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:10:54.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:10:54.865 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:10:54.866 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:10:54.866 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:10:54.869 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:10:54.869 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T17:11:07.493 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:11:07.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:11:07.495 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T17:11:07.495 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:11:07.495 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:11:07.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:11:07.492+0000 7f3431451700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 50492) UID: 1000 2021-02-20T17:11:07.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:11:13.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:11:13.598 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:11:13.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:11:13.599 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:11:13.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:11:13.602 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T17:12:31.871 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:12:31.872 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:12:31.873 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T17:12:31.873 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:12:31.873 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:12:31.874 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T17:12:31.870+0000 7f6d19d2f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 50675) UID: 1000 2021-02-20T17:12:32.074 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:12:37.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:12:37.976 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:12:37.976 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:12:37.976 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:12:37.980 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:12:37.980 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T17:13:06.603 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6305389592054105) > weight (0.25) 2021-02-20T17:13:06.604 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9581432778335826) > weight (0.25) 2021-02-20T17:13:06.604 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5377336865114717) > weight (0.25) 2021-02-20T17:13:06.604 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T17:13:06.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T17:13:06.605 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T17:13:06.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T17:13:06.601+0000 7f03159fd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 49938) UID: 1000 2021-02-20T17:13:06.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:13:10.606 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T17:13:10.607 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:13:10.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:13:10.608 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T17:13:10.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T17:13:10.608 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T17:13:10.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T17:13:10.611 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T17:14:34.981 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5608512112356825) > weight (0.25) 2021-02-20T17:14:34.982 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5685037448219483) > weight (0.25) 2021-02-20T17:14:34.983 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6420528382043872) > weight (0.25) 2021-02-20T17:14:34.983 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T17:14:34.984 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T17:14:34.984 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T17:14:34.985 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T17:14:34.983+0000 7f7f074a3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 47619) UID: 1000 2021-02-20T17:14:35.186 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:14:42.986 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T17:14:42.986 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:14:42.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:14:42.987 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T17:14:42.987 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T17:14:42.988 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T17:14:42.991 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T17:14:42.991 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T17:14:44.612 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3374509332880711) > weight (0.25) 2021-02-20T17:14:44.612 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5179010408940403) > weight (0.25) 2021-02-20T17:14:44.613 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5485710928846333) > weight (0.25) 2021-02-20T17:14:44.613 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5344877740245105) > weight (0.25) 2021-02-20T17:14:44.613 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T17:15:43.993 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8400759557376075) > weight (0.25) 2021-02-20T17:15:43.993 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4822523352369601) > weight (0.25) 2021-02-20T17:15:43.994 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.44401694551385473) > weight (0.25) 2021-02-20T17:15:43.994 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.27170531907444206) > weight (0.25) 2021-02-20T17:15:43.994 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T17:15:57.614 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.39243043506590214) > weight (0.25) 2021-02-20T17:15:57.615 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.36043821414697486) > weight (0.25) 2021-02-20T17:15:57.615 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6101752834702939) > weight (0.25) 2021-02-20T17:15:57.615 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.40661858264656525) > weight (0.25) 2021-02-20T17:15:57.616 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T17:16:45.143 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T17:16:45.140+0000 7fb8259e4700 -1 reset not still connected to 0x5578582489c0 2021-02-20T17:16:45.143 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T17:16:45.140+0000 7fb8259e4700 -1 reset not still connected to 0x557866b05ee0 2021-02-20T17:17:20.996 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5642748675633739) > weight (0.25) 2021-02-20T17:17:20.996 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8936590378925233) > weight (0.25) 2021-02-20T17:17:20.996 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.38071617923525847) > weight (0.25) 2021-02-20T17:17:20.997 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2789418264074862) > weight (0.25) 2021-02-20T17:17:20.997 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T17:17:43.616 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:17:43.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:17:43.618 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T17:17:43.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:17:43.618+0000 7f6b8e37c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 50766) UID: 1000 2021-02-20T17:17:43.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:17:44.619 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:17:44.619 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:17:44.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:17:44.620 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:17:44.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:17:44.621 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:17:44.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:17:44.624 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T17:18:27.998 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8805898352922376) > weight (0.25) 2021-02-20T17:18:27.998 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7205701917902695) > weight (0.25) 2021-02-20T17:18:27.999 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7568414937062439) > weight (0.25) 2021-02-20T17:18:27.999 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5714064888592181) > weight (0.25) 2021-02-20T17:18:27.999 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T17:19:17.625 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9935782118113976) > weight (0.25) 2021-02-20T17:19:17.626 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.25199320406156867) > weight (0.25) 2021-02-20T17:19:17.626 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T17:19:17.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T17:19:17.627 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T17:19:17.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T17:19:17.627+0000 7f2b8bde3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 49558) UID: 1000 2021-02-20T17:19:18.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:19:24.628 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T17:19:24.628 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:19:24.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:19:24.629 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T17:19:24.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T17:19:24.630 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T17:19:24.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T17:19:24.633 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T17:19:59.001 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3250934980503152) > weight (0.25) 2021-02-20T17:19:59.001 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8194845334031058) > weight (0.25) 2021-02-20T17:19:59.001 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T17:19:59.002 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T17:19:59.002 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T17:19:59.003 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T17:19:59.003 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:19:59.004 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T17:19:58.998+0000 7efc5c758700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 50583) UID: 1000 2021-02-20T17:19:59.204 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:20:05.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:20:05.105 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T17:20:05.106 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T17:20:05.106 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T17:20:05.109 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T17:20:05.109 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T17:20:40.634 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5270998989339267) > weight (0.25) 2021-02-20T17:20:40.635 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T17:20:40.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T17:20:40.636 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T17:20:40.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T17:20:40.633+0000 7f8e74cb2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 49837) UID: 1000 2021-02-20T17:20:40.712 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T17:20:40.706+0000 7fb8259e4700 -1 reset not still connected to 0x55785c651cf0 2021-02-20T17:20:41.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:20:42.637 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T17:20:42.637 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:20:42.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:20:42.638 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T17:20:42.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T17:20:42.639 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T17:20:42.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T17:20:42.642 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T17:21:11.110 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2997068903067661) > weight (0.25) 2021-02-20T17:21:11.111 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4621251682901102) > weight (0.25) 2021-02-20T17:21:11.111 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T17:21:11.112 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T17:21:11.112 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T17:21:11.113 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T17:21:11.112+0000 7fb585e84700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 51332) UID: 1000 2021-02-20T17:21:11.313 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:21:16.112 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T17:21:16.113 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:21:16.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:21:16.114 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T17:21:16.114 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T17:21:16.114 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T17:21:16.117 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T17:21:16.118 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T17:21:47.643 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5820156114174138) > weight (0.25) 2021-02-20T17:21:47.644 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7752903709762646) > weight (0.25) 2021-02-20T17:21:47.644 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7204770722483574) > weight (0.25) 2021-02-20T17:21:47.644 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.40225795696867805) > weight (0.25) 2021-02-20T17:21:47.644 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T17:22:54.646 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:22:54.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:22:54.648 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T17:22:54.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:22:54.646+0000 7f8997521700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 51140) UID: 1000 2021-02-20T17:22:54.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:22:57.649 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:22:57.650 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:22:57.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:22:57.651 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:22:57.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:22:57.651 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:22:57.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:22:57.655 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T17:23:15.119 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:23:15.120 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:23:15.120 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T17:23:15.121 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T17:23:15.117+0000 7f256c112700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 50858) UID: 1000 2021-02-20T17:23:15.321 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:23:23.120 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:23:23.121 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:23:23.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:23:23.122 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:23:23.122 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:23:23.122 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:23:23.125 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:23:23.125 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T17:24:53.656 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.552487797052265) > weight (0.25) 2021-02-20T17:24:53.656 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6132029129135397) > weight (0.25) 2021-02-20T17:24:53.657 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T17:24:53.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T17:24:53.658 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T17:24:53.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T17:24:53.654+0000 7ff1f9311700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 51240) UID: 1000 2021-02-20T17:24:53.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:25:02.659 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T17:25:02.660 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:25:02.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:25:02.661 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T17:25:02.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T17:25:02.661 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T17:25:02.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T17:25:02.665 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T17:25:14.126 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9995817120789626) > weight (0.25) 2021-02-20T17:25:14.127 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.33219896561565043) > weight (0.25) 2021-02-20T17:25:14.127 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T17:25:14.128 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T17:25:14.128 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T17:25:14.129 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T17:25:14.126+0000 7f83600ce700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 51522) UID: 1000 2021-02-20T17:25:14.329 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:25:18.129 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T17:25:18.129 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:25:18.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:25:18.130 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T17:25:18.130 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T17:25:18.130 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T17:25:18.133 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T17:25:18.133 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T17:26:46.666 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:26:46.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:26:46.667 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T17:26:46.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:26:46.667+0000 7f0668c43700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 51615) UID: 1000 2021-02-20T17:26:47.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:26:49.669 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:26:49.669 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:26:49.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:26:49.670 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:26:49.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:26:49.671 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:26:49.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:26:49.674 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T17:26:57.134 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.841924780130914) > weight (0.25) 2021-02-20T17:26:57.134 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.46072364311576475) > weight (0.25) 2021-02-20T17:26:57.135 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T17:26:57.135 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T17:26:57.136 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T17:26:57.136 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T17:26:57.135+0000 7f15aed2e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 51892) UID: 1000 2021-02-20T17:26:57.337 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:27:01.137 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T17:27:01.137 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:27:01.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:27:01.138 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T17:27:01.138 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T17:27:01.139 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T17:27:01.141 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T17:27:01.141 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T17:28:24.675 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4606074780635987) > weight (0.25) 2021-02-20T17:28:24.676 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T17:28:24.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T17:28:24.677 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T17:28:24.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T17:28:24.677+0000 7f7ea1f5b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 51423) UID: 1000 2021-02-20T17:28:24.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:28:32.678 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T17:28:32.678 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:28:32.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:28:32.679 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T17:28:32.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T17:28:32.680 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T17:28:32.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T17:28:32.683 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T17:28:38.142 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7087531233249565) > weight (0.25) 2021-02-20T17:28:38.143 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8796731025566769) > weight (0.25) 2021-02-20T17:28:38.143 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T17:28:38.143 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T17:28:38.144 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T17:28:38.145 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T17:28:38.144+0000 7ff97d1d7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 52074) UID: 1000 2021-02-20T17:28:38.345 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:28:40.144 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T17:28:40.145 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:28:40.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:28:40.146 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T17:28:40.146 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T17:28:40.146 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T17:28:40.149 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T17:28:40.149 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T17:29:58.151 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8350057721090426) > weight (0.25) 2021-02-20T17:29:58.160 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T17:29:58.161 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T17:29:58.161 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T17:29:58.162 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T17:29:58.158+0000 7f71b942c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 50400) UID: 1000 2021-02-20T17:29:58.363 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:30:06.163 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T17:30:06.163 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:30:06.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:30:06.164 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T17:30:06.164 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T17:30:06.165 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T17:30:06.167 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T17:30:06.168 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T17:30:23.685 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.47513088931050207) > weight (0.25) 2021-02-20T17:30:23.695 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9588269548815104) > weight (0.25) 2021-02-20T17:30:23.697 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.584146544525535) > weight (0.25) 2021-02-20T17:30:23.698 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8643751830937894) > weight (0.25) 2021-02-20T17:30:23.699 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T17:31:23.700 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9526086913375023) > weight (0.25) 2021-02-20T17:31:23.708 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.39470328770784147) > weight (0.25) 2021-02-20T17:31:23.709 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6232345696177305) > weight (0.25) 2021-02-20T17:31:23.709 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9086987470717355) > weight (0.25) 2021-02-20T17:31:23.709 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T17:32:03.169 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4639413052032737) > weight (0.25) 2021-02-20T17:32:03.176 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6265473882253713) > weight (0.25) 2021-02-20T17:32:03.176 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.45463797783402504) > weight (0.25) 2021-02-20T17:32:03.177 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5658750312460863) > weight (0.25) 2021-02-20T17:32:03.177 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T17:32:50.710 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:32:50.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:32:50.720 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T17:32:50.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:32:50.718+0000 7f844d1bb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 51983) UID: 1000 2021-02-20T17:32:50.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:32:58.721 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:32:58.729 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:32:58.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:32:58.730 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:32:58.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:32:58.731 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:32:58.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:32:58.734 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T17:33:47.178 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9310819466356747) > weight (0.25) 2021-02-20T17:33:47.216 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T17:33:47.217 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T17:33:47.217 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T17:33:47.218 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T17:33:47.216+0000 7f1ca77f7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 52351) UID: 1000 2021-02-20T17:33:47.418 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:33:49.218 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T17:33:49.219 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:33:49.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:33:49.219 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T17:33:49.220 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T17:33:49.220 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T17:33:49.223 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T17:33:49.223 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T17:34:43.250 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T17:34:43.242+0000 7fb8259e4700 -1 reset not still connected to 0x557855c1f330 2021-02-20T17:34:43.250 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T17:34:43.242+0000 7fb8259e4700 -1 reset not still connected to 0x557866b04410 2021-02-20T17:34:52.736 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4650974340329731) > weight (0.25) 2021-02-20T17:34:52.743 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7130417497022826) > weight (0.25) 2021-02-20T17:34:52.743 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5561567772133366) > weight (0.25) 2021-02-20T17:34:52.743 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8107670065150039) > weight (0.25) 2021-02-20T17:34:52.743 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T17:35:19.224 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.26788567563055776) > weight (0.25) 2021-02-20T17:35:19.234 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T17:35:19.235 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T17:35:19.235 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T17:35:19.241 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T17:35:19.238+0000 7f4c06c38700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 52534) UID: 1000 2021-02-20T17:35:19.441 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:35:27.236 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T17:35:27.237 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:35:27.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:35:27.238 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T17:35:27.238 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T17:35:27.239 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T17:35:27.242 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T17:35:27.242 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T17:36:25.744 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.423835243899903) > weight (0.25) 2021-02-20T17:36:25.814 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T17:36:25.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T17:36:25.815 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T17:36:25.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T17:36:25.816+0000 7f83e7b9b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 52167) UID: 1000 2021-02-20T17:36:26.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:36:26.816 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T17:36:26.817 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:36:26.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:36:26.817 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T17:36:26.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T17:36:26.818 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T17:36:26.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T17:36:26.822 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T17:37:09.243 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9501400588324748) > weight (0.25) 2021-02-20T17:37:09.244 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T17:37:09.245 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T17:37:09.245 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T17:37:09.246 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T17:37:09.243+0000 7f9f1a0ed700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 52629) UID: 1000 2021-02-20T17:37:09.446 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:37:15.246 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T17:37:15.247 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:37:15.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:37:15.247 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T17:37:15.248 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T17:37:15.248 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T17:37:15.251 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T17:37:15.251 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T17:37:48.823 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7376624764156295) > weight (0.25) 2021-02-20T17:37:48.832 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6481394338701308) > weight (0.25) 2021-02-20T17:37:48.832 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2965601660822671) > weight (0.25) 2021-02-20T17:37:48.832 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3852615590015036) > weight (0.25) 2021-02-20T17:37:48.832 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T17:38:48.252 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.31210815584729523) > weight (0.25) 2021-02-20T17:38:48.281 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.28190125313442094) > weight (0.25) 2021-02-20T17:38:48.281 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7063428197317699) > weight (0.25) 2021-02-20T17:38:48.281 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T17:38:48.282 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T17:38:48.283 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T17:38:48.284 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T17:38:48.280+0000 7efff27bc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 51042) UID: 1000 2021-02-20T17:38:48.684 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:38:53.284 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T17:38:53.574 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:38:53.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:38:53.575 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T17:38:53.575 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T17:38:53.576 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T17:38:53.579 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T17:38:53.579 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T17:39:14.834 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7353129310365165) > weight (0.25) 2021-02-20T17:39:14.834 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9436042555421134) > weight (0.25) 2021-02-20T17:39:14.835 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6481356144455184) > weight (0.25) 2021-02-20T17:39:14.835 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5902633865606817) > weight (0.25) 2021-02-20T17:39:14.835 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T17:40:12.580 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5015215575279309) > weight (0.25) 2021-02-20T17:40:12.592 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4703619726778364) > weight (0.25) 2021-02-20T17:40:12.592 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3839869804984217) > weight (0.25) 2021-02-20T17:40:12.593 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T17:40:12.593 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T17:40:12.594 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T17:40:12.595 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T17:40:12.590+0000 7f183fb0f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 52902) UID: 1000 2021-02-20T17:40:12.795 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:40:19.595 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T17:40:19.602 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:40:19.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:40:19.603 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T17:40:19.603 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T17:40:19.604 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T17:40:19.607 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T17:40:19.608 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T17:40:30.836 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8035438404126065) > weight (0.25) 2021-02-20T17:40:30.841 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T17:40:30.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T17:40:30.843 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T17:40:30.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T17:40:30.837+0000 7fe0b42a9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 52720) UID: 1000 2021-02-20T17:40:31.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:40:38.843 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T17:40:38.850 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:40:38.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:40:38.851 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T17:40:38.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T17:40:38.852 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T17:40:38.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T17:40:38.856 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T17:41:46.857 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.32917076702345405) > weight (0.25) 2021-02-20T17:41:46.867 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5884923857241465) > weight (0.25) 2021-02-20T17:41:46.867 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T17:41:46.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T17:41:46.868 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T17:41:46.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T17:41:46.867+0000 7f5d90060700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 51801) UID: 1000 2021-02-20T17:41:47.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:41:50.608 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:41:50.609 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:41:50.610 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T17:41:50.610 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T17:41:50.607+0000 7f22891a5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 51706) UID: 1000 2021-02-20T17:41:51.011 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:41:53.611 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:41:53.611 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:41:53.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:41:53.612 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:41:53.612 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:41:53.613 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:41:53.616 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:41:53.616 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T17:41:54.869 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T17:41:54.870 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:41:54.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:41:54.870 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T17:41:54.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T17:41:54.871 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T17:41:54.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T17:41:54.873 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T17:43:11.874 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5879674271600754) > weight (0.25) 2021-02-20T17:43:11.886 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.754203006753538) > weight (0.25) 2021-02-20T17:43:11.886 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T17:43:11.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T17:43:11.887 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T17:43:11.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T17:43:11.885+0000 7f10e7b50700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 53270) UID: 1000 2021-02-20T17:43:12.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:43:18.888 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T17:43:18.888 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:43:18.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:43:18.889 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T17:43:18.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T17:43:18.890 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T17:43:18.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T17:43:18.893 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T17:43:45.617 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:43:45.624 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:43:45.625 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T17:43:45.625 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T17:43:45.624+0000 7fd823055700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 53179) UID: 1000 2021-02-20T17:43:45.826 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:43:46.625 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:43:46.626 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:43:46.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:43:46.627 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:43:46.627 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:43:46.627 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:43:46.630 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:43:46.630 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T17:44:28.894 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3470841868441752) > weight (0.25) 2021-02-20T17:44:28.902 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2806400151151327) > weight (0.25) 2021-02-20T17:44:28.902 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.46514878105741064) > weight (0.25) 2021-02-20T17:44:28.902 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T17:44:28.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T17:44:28.903 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T17:44:28.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T17:44:28.903+0000 7f7295a80700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 50949) UID: 1000 2021-02-20T17:44:29.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:44:33.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:44:33.191+0000 7f95d37bb700 -1 rbd::mirror::ImageReplayer: 0x55d10c048800 [3/49a0b5eb-7d69-4824-806b-1ece12354eb9] start: already running 2021-02-20T17:44:33.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:44:33.191+0000 7f95d37bb700 -1 rbd::mirror::ImageReplayer: 0x55d10c048300 [3/3c6940ee-a642-422e-aafb-e3920bba87cf] start: already running 2021-02-20T17:44:33.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:44:33.191+0000 7f95d37bb700 -1 rbd::mirror::ImageReplayer: 0x55d10c048580 [3/3d38caf8-6420-4f62-95dc-628caa5144da] start: already running 2021-02-20T17:44:33.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:44:33.191+0000 7f95d37bb700 -1 rbd::mirror::ImageReplayer: 0x55d10c048800 [3/49a0b5eb-7d69-4824-806b-1ece12354eb9] start: already running 2021-02-20T17:44:33.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:44:33.191+0000 7f95d37bb700 -1 rbd::mirror::ImageReplayer: 0x55d10c048a80 [3/4c862af6-d1da-4215-b6b8-461f3883b104] start: already running 2021-02-20T17:44:33.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:44:33.191+0000 7f95d37bb700 -1 rbd::mirror::ImageReplayer: 0x55d10c048d00 [3/4cba354e-99d2-4859-a4d0-baeec2203656] start: already running 2021-02-20T17:44:33.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:44:33.191+0000 7f95d37bb700 -1 rbd::mirror::ImageReplayer: 0x55d10c048f80 [3/567ace7d-44e4-4d42-b47a-38bd35ec0e11] start: already running 2021-02-20T17:44:33.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:44:33.191+0000 7f95d37bb700 -1 rbd::mirror::ImageReplayer: 0x55d10c049200 [3/5786ec48-0da5-470d-b6a6-ddd71a666657] start: already running 2021-02-20T17:44:33.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:44:33.191+0000 7f95d37bb700 -1 rbd::mirror::ImageReplayer: 0x55d10c049480 [3/5c036645-b242-4fa8-bde0-d5b28dda3fd3] start: already running 2021-02-20T17:44:33.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:44:33.191+0000 7f95d37bb700 -1 rbd::mirror::ImageReplayer: 0x55d10c049700 [3/5f3204b5-90c6-4244-a940-e749860457d5] start: already running 2021-02-20T17:44:33.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:44:33.191+0000 7f95d37bb700 -1 rbd::mirror::ImageReplayer: 0x55d10c049980 [3/675a1b7b-4e17-4b1e-9e92-eaf406f1c33f] start: already running 2021-02-20T17:44:33.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:44:33.191+0000 7f95d37bb700 -1 rbd::mirror::ImageReplayer: 0x55d10c049c00 [3/681dbea5-7557-4dba-a92d-01da85613e31] start: already running 2021-02-20T17:44:33.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:44:33.191+0000 7f95d37bb700 -1 rbd::mirror::ImageReplayer: 0x55d110110000 [3/753c14e7-97d4-4109-8814-2c7f1abbbee8] start: already running 2021-02-20T17:44:33.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:44:33.191+0000 7f95d37bb700 -1 rbd::mirror::ImageReplayer: 0x55d110110280 [3/7658e9e0-5116-4c47-bad5-4e7894e27da7] start: already running 2021-02-20T17:44:33.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:44:33.191+0000 7f95d37bb700 -1 rbd::mirror::ImageReplayer: 0x55d110110500 [3/790c038c-4353-4ff1-ae0b-9c9bb1e1adaa] start: already running 2021-02-20T17:44:33.905 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T17:44:33.909 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:44:33.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:44:33.910 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T17:44:33.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T17:44:33.911 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T17:44:33.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T17:44:33.914 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T17:45:36.632 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3608441011062212) > weight (0.25) 2021-02-20T17:45:36.642 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T17:45:36.643 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T17:45:36.644 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T17:45:36.645 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T17:45:36.641+0000 7faecd88e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 52811) UID: 1000 2021-02-20T17:45:36.845 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:45:44.645 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T17:45:44.645 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:45:44.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:45:44.646 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T17:45:44.646 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T17:45:44.647 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T17:45:44.657 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T17:45:44.657 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T17:45:49.853 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T17:45:49.849+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a6d6f740 2021-02-20T17:45:49.855 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T17:45:49.849+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ac75b380 2021-02-20T17:45:49.855 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T17:45:49.849+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b8a8e340 2021-02-20T17:45:49.862 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T17:45:49.853+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6cfb670c0 2021-02-20T17:46:20.914 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8692097344976153) > weight (0.25) 2021-02-20T17:46:20.915 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.992021456113699) > weight (0.25) 2021-02-20T17:46:20.915 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8794201021650998) > weight (0.25) 2021-02-20T17:46:20.915 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.36959083432394835) > weight (0.25) 2021-02-20T17:46:20.915 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T17:47:10.657 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:47:10.667 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:47:10.667 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T17:47:10.668 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T17:47:10.667+0000 7f7d06e29700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 53452) UID: 1000 2021-02-20T17:47:10.869 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:47:14.668 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:47:14.669 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:47:14.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:47:14.670 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:47:14.670 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:47:14.670 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:47:14.673 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:47:14.674 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T17:48:03.916 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8896791604316305) > weight (0.25) 2021-02-20T17:48:03.925 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9288941941364228) > weight (0.25) 2021-02-20T17:48:03.925 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.600382424695397) > weight (0.25) 2021-02-20T17:48:03.925 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5639708383723893) > weight (0.25) 2021-02-20T17:48:03.926 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T17:48:25.675 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9222680659095293) > weight (0.25) 2021-02-20T17:48:25.680 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5433164631730741) > weight (0.25) 2021-02-20T17:48:25.681 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.48103390989188566) > weight (0.25) 2021-02-20T17:48:25.681 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5613644329582755) > weight (0.25) 2021-02-20T17:48:25.681 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T17:49:39.926 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7412607442254459) > weight (0.25) 2021-02-20T17:49:39.937 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7592346610499331) > weight (0.25) 2021-02-20T17:49:39.937 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9844340740587472) > weight (0.25) 2021-02-20T17:49:39.938 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6727349614260155) > weight (0.25) 2021-02-20T17:49:39.938 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T17:50:13.682 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8809470411004552) > weight (0.25) 2021-02-20T17:50:13.694 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8670276290682069) > weight (0.25) 2021-02-20T17:50:13.695 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4011390472245321) > weight (0.25) 2021-02-20T17:50:13.695 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.797866184528682) > weight (0.25) 2021-02-20T17:50:13.695 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T17:51:17.697 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8659757017148046) > weight (0.25) 2021-02-20T17:51:17.706 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6562751565323766) > weight (0.25) 2021-02-20T17:51:17.706 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7395480359336546) > weight (0.25) 2021-02-20T17:51:17.707 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9998179072725983) > weight (0.25) 2021-02-20T17:51:17.707 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T17:51:25.939 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4262966856469208) > weight (0.25) 2021-02-20T17:51:25.948 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8922560500844904) > weight (0.25) 2021-02-20T17:51:25.949 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8764886334004403) > weight (0.25) 2021-02-20T17:51:25.949 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7658579425128784) > weight (0.25) 2021-02-20T17:51:25.949 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T17:52:28.707 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5693053530279876) > weight (0.25) 2021-02-20T17:52:28.715 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6189216740712767) > weight (0.25) 2021-02-20T17:52:28.715 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5748899864617983) > weight (0.25) 2021-02-20T17:52:28.716 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9773996330106527) > weight (0.25) 2021-02-20T17:52:28.716 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T17:52:53.950 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.47713925681492697) > weight (0.25) 2021-02-20T17:52:53.959 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T17:52:53.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T17:52:53.960 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T17:52:53.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T17:52:53.958+0000 7fa1447df700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 53088) UID: 1000 2021-02-20T17:52:54.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:52:54.961 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T17:52:54.962 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:52:54.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:52:54.963 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T17:52:54.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T17:52:54.964 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T17:52:54.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T17:52:54.967 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T17:54:20.968 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6254826899516815) > weight (0.25) 2021-02-20T17:54:20.975 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T17:54:20.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T17:54:20.977 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T17:54:20.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T17:54:20.975+0000 7fa352a00700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 53833) UID: 1000 2021-02-20T17:54:21.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:54:23.716 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8793827882323301) > weight (0.25) 2021-02-20T17:54:23.717 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5415379776733082) > weight (0.25) 2021-02-20T17:54:23.717 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7578141984642929) > weight (0.25) 2021-02-20T17:54:23.717 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.40543321610639227) > weight (0.25) 2021-02-20T17:54:23.717 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T17:54:24.978 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T17:54:24.978 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:54:24.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:54:24.979 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T17:54:24.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T17:54:24.980 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T17:54:24.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T17:54:24.983 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T17:55:51.718 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5137168189483315) > weight (0.25) 2021-02-20T17:55:51.728 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T17:55:51.729 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T17:55:51.729 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T17:55:51.730 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T17:55:51.725+0000 7f6738bfa700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 53647) UID: 1000 2021-02-20T17:55:51.931 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:55:52.731 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T17:55:52.731 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:55:52.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:55:52.732 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T17:55:52.733 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T17:55:52.733 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T17:55:52.736 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T17:55:52.736 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T17:56:11.984 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.634624968471883) > weight (0.25) 2021-02-20T17:56:11.985 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2566288444437499) > weight (0.25) 2021-02-20T17:56:11.985 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7733753981927824) > weight (0.25) 2021-02-20T17:56:11.985 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T17:56:11.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T17:56:11.986 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T17:56:11.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T17:56:11.984+0000 7f9602464700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 53552) UID: 1000 2021-02-20T17:56:12.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:56:20.987 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T17:56:20.988 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:56:20.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:56:20.989 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T17:56:20.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T17:56:20.989 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T17:56:20.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T17:56:20.993 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T17:57:46.738 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6881356347946258) > weight (0.25) 2021-02-20T17:57:46.739 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T17:57:46.740 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T17:57:46.741 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T17:57:46.741 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T17:57:46.738+0000 7fb41b969700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 54020) UID: 1000 2021-02-20T17:57:46.942 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:57:55.741 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T17:57:55.742 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:57:55.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:57:55.743 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T17:57:55.743 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T17:57:55.744 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T17:57:55.746 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T17:57:55.747 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T17:58:17.993 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8615220088816621) > weight (0.25) 2021-02-20T17:58:17.994 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6624178085588359) > weight (0.25) 2021-02-20T17:58:17.994 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.959829709910709) > weight (0.25) 2021-02-20T17:58:17.994 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5627404938913595) > weight (0.25) 2021-02-20T17:58:17.995 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T17:59:02.748 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:59:02.749 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:59:02.750 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T17:59:02.751 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T17:59:02.748+0000 7f85c13eb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 53738) UID: 1000 2021-02-20T17:59:02.951 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:59:04.751 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:59:04.751 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:59:04.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:59:04.752 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:59:04.752 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:59:04.753 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:59:04.755 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:59:04.756 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T17:59:26.996 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:59:26.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:59:26.997 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T17:59:26.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T17:59:26.995+0000 7f95d47bd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 52442) UID: 1000 2021-02-20T17:59:27.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:59:28.998 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:59:28.998 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:59:28.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:59:28.999 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:59:28.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:59:28.999 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:59:29.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:59:29.003 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T17:59:49.891 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T17:59:49.886+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a6d6f740 2021-02-20T17:59:49.891 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T17:59:49.886+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ac75b380 2021-02-20T17:59:49.892 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T17:59:49.886+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b8a8e340 2021-02-20T17:59:49.892 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T17:59:49.886+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6cfb670c0 2021-02-20T18:00:18.756 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7039728181813314) > weight (0.25) 2021-02-20T18:00:18.757 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6994005997424556) > weight (0.25) 2021-02-20T18:00:18.758 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T18:00:18.758 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T18:00:18.759 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T18:00:18.760 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T18:00:18.754+0000 7f14d02a8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 52258) UID: 1000 2021-02-20T18:00:18.960 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:00:20.760 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T18:00:20.761 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:00:20.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:00:20.761 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T18:00:20.762 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T18:00:20.762 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T18:00:20.765 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T18:00:20.765 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T18:00:57.004 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.83737742724347) > weight (0.25) 2021-02-20T18:00:57.004 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T18:00:57.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T18:00:57.006 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T18:00:57.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T18:00:57.005+0000 7f4140b4c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 53924) UID: 1000 2021-02-20T18:00:57.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:01:03.007 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T18:01:03.007 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:01:03.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:01:03.008 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T18:01:03.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T18:01:03.009 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T18:01:03.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T18:01:03.012 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T18:01:44.766 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5134254102048651) > weight (0.25) 2021-02-20T18:01:44.777 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T18:01:44.778 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T18:01:44.779 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T18:01:44.780 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T18:01:44.775+0000 7fc27567a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 54219) UID: 1000 2021-02-20T18:01:44.980 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:01:45.780 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T18:01:45.780 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:01:45.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:01:45.781 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T18:01:45.782 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T18:01:45.782 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T18:01:45.785 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T18:01:45.785 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T18:02:29.014 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:02:29.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:02:29.016 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T18:02:29.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:02:29.014+0000 7f7ecb250700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 54401) UID: 1000 2021-02-20T18:02:29.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:02:34.016 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:02:34.017 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:02:34.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:02:34.018 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:02:34.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:02:34.018 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:02:34.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:02:34.022 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T18:03:16.785 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8451660411519434) > weight (0.25) 2021-02-20T18:03:16.786 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2954235022350299) > weight (0.25) 2021-02-20T18:03:16.786 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9796058784100248) > weight (0.25) 2021-02-20T18:03:16.787 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.34837900509033815) > weight (0.25) 2021-02-20T18:03:16.787 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T18:04:24.023 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.588523101286338) > weight (0.25) 2021-02-20T18:04:24.024 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5173722232598573) > weight (0.25) 2021-02-20T18:04:24.024 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3124473187648962) > weight (0.25) 2021-02-20T18:04:24.024 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.43153511099318376) > weight (0.25) 2021-02-20T18:04:24.025 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T18:05:12.788 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9965375449993477) > weight (0.25) 2021-02-20T18:05:12.789 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8795650689667814) > weight (0.25) 2021-02-20T18:05:12.789 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T18:05:12.790 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T18:05:12.791 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T18:05:12.792 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T18:05:12.790+0000 7f5b01e39700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 54569) UID: 1000 2021-02-20T18:05:12.992 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:05:20.792 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T18:05:20.792 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:05:20.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:05:20.793 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T18:05:20.793 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T18:05:20.794 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T18:05:20.797 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T18:05:20.797 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T18:05:37.026 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5288968213032915) > weight (0.25) 2021-02-20T18:05:37.026 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9324181530037516) > weight (0.25) 2021-02-20T18:05:37.027 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8596050955263731) > weight (0.25) 2021-02-20T18:05:37.027 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9454175079812593) > weight (0.25) 2021-02-20T18:05:37.027 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T18:06:19.873 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T18:06:19.868+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a6d6f740 2021-02-20T18:06:19.874 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T18:06:19.868+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ac75b380 2021-02-20T18:06:19.874 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T18:06:19.868+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b8a8e340 2021-02-20T18:06:19.874 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T18:06:19.868+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6cfb670c0 2021-02-20T18:07:04.028 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5730388698787591) > weight (0.25) 2021-02-20T18:07:04.029 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T18:07:04.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T18:07:04.030 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T18:07:04.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T18:07:04.027+0000 7f1231191700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 54669) UID: 1000 2021-02-20T18:07:04.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:07:05.798 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6504444726232419) > weight (0.25) 2021-02-20T18:07:05.798 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.41461881028636616) > weight (0.25) 2021-02-20T18:07:05.799 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.42925496748612935) > weight (0.25) 2021-02-20T18:07:05.799 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8244951649020399) > weight (0.25) 2021-02-20T18:07:05.799 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T18:07:07.031 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T18:07:07.032 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:07:07.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:07:07.033 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T18:07:07.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T18:07:07.033 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T18:07:07.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T18:07:07.037 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T18:08:21.038 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9475728457296193) > weight (0.25) 2021-02-20T18:08:21.039 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4329821250562348) > weight (0.25) 2021-02-20T18:08:21.039 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4143310352144217) > weight (0.25) 2021-02-20T18:08:21.039 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6293343105699097) > weight (0.25) 2021-02-20T18:08:21.040 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T18:08:31.800 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5703487546593438) > weight (0.25) 2021-02-20T18:08:31.801 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5683877964721442) > weight (0.25) 2021-02-20T18:08:31.801 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6374685751151009) > weight (0.25) 2021-02-20T18:08:31.801 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3904720132799566) > weight (0.25) 2021-02-20T18:08:31.802 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T18:09:53.041 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.904190296087926) > weight (0.25) 2021-02-20T18:09:53.042 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6723538020841904) > weight (0.25) 2021-02-20T18:09:53.042 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8126223216486329) > weight (0.25) 2021-02-20T18:09:53.043 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.2600838568596737) > weight (0.25) 2021-02-20T18:09:53.043 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T18:09:57.802 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.46579054664185227) > weight (0.25) 2021-02-20T18:09:57.803 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.30400325361459246) > weight (0.25) 2021-02-20T18:09:57.803 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8170851390707334) > weight (0.25) 2021-02-20T18:09:57.803 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9931424069751659) > weight (0.25) 2021-02-20T18:09:57.803 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T18:11:24.805 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.34439942034906756) > weight (0.25) 2021-02-20T18:11:24.806 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5939653406269415) > weight (0.25) 2021-02-20T18:11:24.806 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5210426767898884) > weight (0.25) 2021-02-20T18:11:24.807 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.570482549611076) > weight (0.25) 2021-02-20T18:11:24.807 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T18:11:35.044 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7029112323662524) > weight (0.25) 2021-02-20T18:11:35.044 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T18:11:35.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T18:11:35.046 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T18:11:35.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T18:11:35.044+0000 7fa17dde6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 55040) UID: 1000 2021-02-20T18:11:35.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:11:43.047 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T18:11:43.047 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:11:43.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:11:43.048 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T18:11:43.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T18:11:43.049 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T18:11:43.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T18:11:43.052 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T18:12:32.808 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7062996346990636) > weight (0.25) 2021-02-20T18:12:32.809 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5463234625514155) > weight (0.25) 2021-02-20T18:12:32.809 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9668860689788527) > weight (0.25) 2021-02-20T18:12:32.809 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8463653438214107) > weight (0.25) 2021-02-20T18:12:32.810 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T18:13:27.053 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.39914137965336505) > weight (0.25) 2021-02-20T18:13:27.054 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T18:13:27.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T18:13:27.056 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T18:13:27.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T18:13:27.057+0000 7f9448c2e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 55133) UID: 1000 2021-02-20T18:13:27.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:13:36.057 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T18:13:36.057 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:13:36.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:13:36.058 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T18:13:36.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T18:13:36.059 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T18:13:36.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T18:13:36.063 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T18:13:42.810 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4063085012132758) > weight (0.25) 2021-02-20T18:13:42.811 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6573156089441364) > weight (0.25) 2021-02-20T18:13:42.811 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9004215128144692) > weight (0.25) 2021-02-20T18:13:42.811 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T18:13:42.812 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T18:13:42.812 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T18:13:42.813 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T18:13:42.812+0000 7f843f2c8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 52997) UID: 1000 2021-02-20T18:13:43.013 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:13:43.814 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T18:13:43.815 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:13:43.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:13:43.816 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T18:13:43.816 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T18:13:43.816 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T18:13:43.819 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T18:13:43.819 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T18:13:47.065 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T18:13:47.060+0000 7ff44d0aa700 -1 rbd::mirror::ImageReplayer: 0x55eb59b8b200 [3/49a0b5eb-7d69-4824-806b-1ece12354eb9] start: already running 2021-02-20T18:13:47.076 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T18:13:47.072+0000 7ff44c8a9700 -1 rbd::mirror::ImageReplayer: 0x55eb5dc8b400 [3/ff38d3a1-22f6-4a7c-a814-2d4505cf6570] start: already running 2021-02-20T18:14:59.821 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:14:59.822 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:14:59.823 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T18:14:59.824 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T18:14:59.822+0000 7ff44d8ab700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 54310) UID: 1000 2021-02-20T18:15:00.024 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:15:07.823 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:15:07.824 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:15:07.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:15:07.825 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:15:07.825 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:15:07.826 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:15:07.829 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:15:07.829 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T18:15:23.064 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7348839384680917) > weight (0.25) 2021-02-20T18:15:23.064 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.984401606890813) > weight (0.25) 2021-02-20T18:15:23.065 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T18:15:23.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T18:15:23.066 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T18:15:23.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T18:15:23.066+0000 7fa4b6267700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 53361) UID: 1000 2021-02-20T18:15:23.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:15:26.066 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T18:15:26.067 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:15:26.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:15:26.068 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T18:15:26.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T18:15:26.068 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T18:15:26.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T18:15:26.071 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T18:16:29.830 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:16:29.832 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:16:29.832 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T18:16:29.833 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T18:16:29.832+0000 7f8714052700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 55410) UID: 1000 2021-02-20T18:16:30.033 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:16:37.833 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:16:37.834 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:16:37.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:16:37.835 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:16:37.835 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:16:37.836 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:16:37.840 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:16:37.840 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T18:16:42.072 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7183138145279723) > weight (0.25) 2021-02-20T18:16:42.072 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6946529129306664) > weight (0.25) 2021-02-20T18:16:42.073 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8399144339666395) > weight (0.25) 2021-02-20T18:16:42.073 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T18:16:42.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T18:16:42.075 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T18:16:42.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T18:16:42.071+0000 7f0da4a3d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 54111) UID: 1000 2021-02-20T18:16:42.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:16:48.076 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T18:16:48.077 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:16:48.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:16:48.077 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T18:16:48.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T18:16:48.078 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T18:16:48.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T18:16:48.081 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T18:18:25.842 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:18:25.843 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:18:25.844 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T18:18:25.845 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T18:18:25.841+0000 7f1d09880700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 55593) UID: 1000 2021-02-20T18:18:26.045 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:18:30.845 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:18:30.845 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:18:30.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:18:30.846 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:18:30.846 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:18:30.847 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:18:30.849 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:18:30.850 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T18:18:35.082 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.366666772559877) > weight (0.25) 2021-02-20T18:18:35.082 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7184586492464454) > weight (0.25) 2021-02-20T18:18:35.082 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5634906098426097) > weight (0.25) 2021-02-20T18:18:35.082 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.27043355685712955) > weight (0.25) 2021-02-20T18:18:35.083 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T18:19:51.083 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7950147008114685) > weight (0.25) 2021-02-20T18:19:51.084 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6435331294243894) > weight (0.25) 2021-02-20T18:19:51.084 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7600873477779635) > weight (0.25) 2021-02-20T18:19:51.085 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9987632326559619) > weight (0.25) 2021-02-20T18:19:51.085 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T18:20:20.851 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:20:20.852 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:20:20.853 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T18:20:20.854 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T18:20:20.850+0000 7fb61c973700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 55787) UID: 1000 2021-02-20T18:20:21.054 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:20:25.854 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:20:25.854 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:20:25.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:20:25.855 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:20:25.855 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:20:25.856 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:20:25.858 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:20:25.859 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T18:21:31.086 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7984305022063882) > weight (0.25) 2021-02-20T18:21:31.086 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T18:21:31.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T18:21:31.088 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T18:21:31.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T18:21:31.088+0000 7f470dd0d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 55225) UID: 1000 2021-02-20T18:21:31.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:21:35.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:21:35.400+0000 7f0b09bb2700 -1 rbd::mirror::ImageReplayer: 0x55baf497d980 [3/e5f43e4f-211a-45bf-bf8b-2e986b46e688] start: already running 2021-02-20T18:21:35.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:21:35.400+0000 7f0b09bb2700 -1 rbd::mirror::ImageReplayer: 0x55baf497dc00 [3/eafa574f-c2c8-4659-b559-be75ec7d56bb] start: already running 2021-02-20T18:21:35.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:21:35.400+0000 7f0b09bb2700 -1 rbd::mirror::ImageReplayer: 0x55baf49f0000 [3/ef164021-65e4-434b-a5d7-bcdd53df22fd] start: already running 2021-02-20T18:21:35.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:21:35.400+0000 7f0b09bb2700 -1 rbd::mirror::ImageReplayer: 0x55baf49f0280 [3/ff38d3a1-22f6-4a7c-a814-2d4505cf6570] start: already running 2021-02-20T18:21:35.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:21:35.808+0000 7f0b093b1700 -1 rbd::mirror::ImageReplayer: 0x55baf497c800 [3/bbf1cffa-5e93-4c1c-937d-71c8c54806ff] register_admin_socket_hook: error registering admin socket commands 2021-02-20T18:21:35.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:21:35.808+0000 7f0b09bb2700 -1 rbd::mirror::ImageReplayer: 0x55baf497c800 [3/bbf1cffa-5e93-4c1c-937d-71c8c54806ff] register_admin_socket_hook: error registering admin socket commands 2021-02-20T18:21:35.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:21:35.808+0000 7f0b093b1700 -1 rbd::mirror::ImageReplayer: 0x55baf497c580 [3/b97c3e7d-cef7-4890-a315-ffa1f7e51804] register_admin_socket_hook: error registering admin socket commands 2021-02-20T18:21:35.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:21:35.808+0000 7f0b09bb2700 -1 rbd::mirror::ImageReplayer: 0x55baf497c580 [3/b97c3e7d-cef7-4890-a315-ffa1f7e51804] register_admin_socket_hook: error registering admin socket commands 2021-02-20T18:21:35.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:21:35.876+0000 7f0b093b1700 -1 rbd::mirror::ImageReplayer: 0x55baf497ac80 [3/941323f6-0453-4944-ab79-664fde6ec580] register_admin_socket_hook: error registering admin socket commands 2021-02-20T18:21:35.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:21:35.876+0000 7f0b09bb2700 -1 rbd::mirror::ImageReplayer: 0x55baf497ac80 [3/941323f6-0453-4944-ab79-664fde6ec580] register_admin_socket_hook: error registering admin socket commands 2021-02-20T18:21:35.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:21:35.876+0000 7f0b093b1700 -1 rbd::mirror::ImageReplayer: 0x55baf497a780 [3/8c216315-5117-4846-b850-c4265b4697a3] register_admin_socket_hook: error registering admin socket commands 2021-02-20T18:21:35.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:21:35.876+0000 7f0b09bb2700 -1 rbd::mirror::ImageReplayer: 0x55baf497a780 [3/8c216315-5117-4846-b850-c4265b4697a3] register_admin_socket_hook: error registering admin socket commands 2021-02-20T18:21:40.088 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T18:21:40.089 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:21:40.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:21:40.089 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T18:21:40.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T18:21:40.090 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T18:21:40.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T18:21:40.095 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T18:22:03.860 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5506404009461379) > weight (0.25) 2021-02-20T18:22:03.861 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.813026592830056) > weight (0.25) 2021-02-20T18:22:03.861 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T18:22:03.862 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T18:22:03.862 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T18:22:03.863 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T18:22:03.859+0000 7f676c2b6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 54946) UID: 1000 2021-02-20T18:22:04.063 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:22:08.863 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T18:22:08.864 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:22:08.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:22:08.865 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T18:22:08.865 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T18:22:08.865 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T18:22:08.868 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T18:22:08.868 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T18:23:01.096 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4733775699565077) > weight (0.25) 2021-02-20T18:23:01.097 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8687753734938969) > weight (0.25) 2021-02-20T18:23:01.097 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9433950677975835) > weight (0.25) 2021-02-20T18:23:01.097 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7988870748667913) > weight (0.25) 2021-02-20T18:23:01.097 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T18:23:42.869 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6951005915355776) > weight (0.25) 2021-02-20T18:23:42.877 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T18:23:42.878 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T18:23:42.878 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T18:23:42.879 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T18:23:42.876+0000 7f32acf88700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 54760) UID: 1000 2021-02-20T18:23:43.280 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:23:47.879 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T18:23:47.880 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:23:47.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:23:47.881 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T18:23:47.881 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T18:23:47.881 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T18:23:47.886 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T18:23:47.887 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T18:24:48.098 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:24:48.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:24:48.108 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T18:24:48.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:24:48.103+0000 7f0b0a3b3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 54852) UID: 1000 2021-02-20T18:24:48.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:24:52.108 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:24:52.109 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:24:52.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:24:52.110 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:24:52.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:24:52.110 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:24:52.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:24:52.113 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T18:25:38.888 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2691270891322083) > weight (0.25) 2021-02-20T18:25:38.888 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T18:25:38.889 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T18:25:38.889 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T18:25:38.889 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T18:25:38.890 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:25:38.890 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T18:25:38.885+0000 7fc5cd32f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 56165) UID: 1000 2021-02-20T18:25:39.090 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:25:44.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:25:44.992 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T18:25:44.992 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T18:25:44.992 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T18:25:44.995 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T18:25:44.995 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T18:26:23.114 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6285310850142535) > weight (0.25) 2021-02-20T18:26:23.115 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7565853990814153) > weight (0.25) 2021-02-20T18:26:23.115 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3258338255967712) > weight (0.25) 2021-02-20T18:26:23.115 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.2930181832213218) > weight (0.25) 2021-02-20T18:26:23.115 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T18:27:29.997 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:27:29.998 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:27:29.999 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T18:27:30.000 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T18:27:29.994+0000 7f28999fc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 55880) UID: 1000 2021-02-20T18:27:30.200 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:27:34.275 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T18:27:34.270+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6bab4dfb0 2021-02-20T18:27:34.276 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T18:27:34.270+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6bab4fe90 2021-02-20T18:27:34.276 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T18:27:34.270+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6e58ac970 2021-02-20T18:27:34.276 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T18:27:34.270+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6e58ace50 2021-02-20T18:27:35.999 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:27:35.999 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:27:36.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:27:36.000 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:27:36.000 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:27:36.001 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:27:36.004 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:27:36.004 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T18:28:10.117 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:28:10.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:28:10.118 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T18:28:10.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:28:10.113+0000 7fa3b5715700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 56256) UID: 1000 2021-02-20T18:28:10.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:28:12.120 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:28:12.120 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:28:12.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:28:12.121 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:28:12.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:28:12.122 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:28:12.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:28:12.125 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T18:29:27.005 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.884284448742812) > weight (0.25) 2021-02-20T18:29:27.016 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.33413545488698626) > weight (0.25) 2021-02-20T18:29:27.017 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T18:29:27.017 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T18:29:27.018 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T18:29:27.019 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T18:29:27.015+0000 7f25df8a0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 56072) UID: 1000 2021-02-20T18:29:27.219 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:29:31.019 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T18:29:31.020 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:29:31.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:29:31.020 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T18:29:31.021 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T18:29:31.021 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T18:29:31.024 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T18:29:31.024 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T18:30:08.125 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7558495562819152) > weight (0.25) 2021-02-20T18:30:08.137 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7541536030374573) > weight (0.25) 2021-02-20T18:30:08.138 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.675025274118802) > weight (0.25) 2021-02-20T18:30:08.138 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7273616568688966) > weight (0.25) 2021-02-20T18:30:08.138 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T18:30:44.105 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T18:30:44.097+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6bab4dfb0 2021-02-20T18:30:44.106 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T18:30:44.097+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6bab4fe90 2021-02-20T18:30:44.106 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T18:30:44.097+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6e58ac970 2021-02-20T18:30:44.106 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T18:30:44.097+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6e58ace50 2021-02-20T18:30:52.025 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8748118225736063) > weight (0.25) 2021-02-20T18:30:55.921 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T18:30:55.921 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T18:30:55.922 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T18:30:55.923 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T18:30:55.921+0000 7fabdfba5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 56358) UID: 1000 2021-02-20T18:30:56.123 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:31:01.923 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T18:31:01.923 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:31:01.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:31:01.924 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T18:31:01.924 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T18:31:01.925 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T18:31:01.928 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T18:31:01.928 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T18:31:51.139 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:31:51.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:31:51.141 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T18:31:51.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:31:51.135+0000 7fc53c030700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 56541) UID: 1000 2021-02-20T18:31:51.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:31:55.142 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:31:55.152 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:31:55.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:31:55.153 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:31:55.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:31:55.154 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:31:55.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:31:55.158 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T18:32:48.929 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6622677077151714) > weight (0.25) 2021-02-20T18:32:48.945 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5326259432533513) > weight (0.25) 2021-02-20T18:32:48.945 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T18:32:48.946 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T18:32:48.946 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T18:32:48.946 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T18:32:48.947 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:32:48.950 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T18:32:48.946+0000 7f29f95c9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 56634) UID: 1000 2021-02-20T18:32:49.150 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:32:55.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:32:55.050 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T18:32:55.050 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T18:32:55.051 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T18:32:55.054 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T18:32:55.055 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T18:33:37.160 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:33:37.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:33:37.161 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T18:33:37.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:33:37.160+0000 7fe08928d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 56818) UID: 1000 2021-02-20T18:33:37.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:33:45.162 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:33:45.163 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:33:45.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:33:45.163 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:33:45.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:33:45.164 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:33:45.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:33:45.167 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T18:33:59.051 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T18:33:59.048+0000 7f4a1bb72700 -1 reset not still connected to 0x55fc7bc315f0 2021-02-20T18:34:53.056 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6701283664923425) > weight (0.25) 2021-02-20T18:34:53.057 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9515886579049557) > weight (0.25) 2021-02-20T18:34:53.057 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9068261763364809) > weight (0.25) 2021-02-20T18:34:53.057 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T18:34:53.058 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T18:34:53.058 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T18:34:53.059 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T18:34:53.054+0000 7f57d8415700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 55316) UID: 1000 2021-02-20T18:34:53.168 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4781256130646464) > weight (0.25) 2021-02-20T18:34:53.168 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5241767351844817) > weight (0.25) 2021-02-20T18:34:53.169 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T18:34:53.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T18:34:53.169 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T18:34:53.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T18:34:53.166+0000 7fe633693700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 55501) UID: 1000 2021-02-20T18:34:53.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:34:53.460 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:34:58.170 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T18:34:58.171 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:34:58.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:34:58.171 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T18:34:58.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T18:34:58.172 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T18:34:58.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T18:34:58.175 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T18:35:01.059 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T18:35:01.060 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:35:01.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:35:01.061 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T18:35:01.061 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T18:35:01.061 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T18:35:01.064 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T18:35:01.064 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T18:35:06.267 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T18:35:06.262+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6bab4dd40 2021-02-20T18:35:06.270 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T18:35:06.266+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ca080a40 2021-02-20T18:36:37.065 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6040653740691033) > weight (0.25) 2021-02-20T18:36:37.078 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6472946276325798) > weight (0.25) 2021-02-20T18:36:37.078 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T18:36:37.079 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T18:36:37.079 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T18:36:37.080 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T18:36:37.076+0000 7fe5514d4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 56909) UID: 1000 2021-02-20T18:36:37.281 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:36:38.080 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T18:36:38.081 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:36:38.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:36:38.082 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T18:36:38.082 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T18:36:38.082 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T18:36:38.085 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T18:36:38.085 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T18:36:39.177 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:36:39.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:36:39.178 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T18:36:39.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:36:39.175+0000 7fee7f191700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 57002) UID: 1000 2021-02-20T18:36:39.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:36:43.178 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:36:43.179 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:36:43.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:36:43.179 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:36:43.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:36:43.180 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:36:43.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:36:43.183 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T18:37:51.086 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6790584880084627) > weight (0.25) 2021-02-20T18:37:51.094 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3316657265734695) > weight (0.25) 2021-02-20T18:37:51.094 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4607181838977229) > weight (0.25) 2021-02-20T18:37:51.094 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4697859159353034) > weight (0.25) 2021-02-20T18:37:51.095 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T18:38:29.184 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8850234301250909) > weight (0.25) 2021-02-20T18:38:29.191 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5082156985401345) > weight (0.25) 2021-02-20T18:38:29.192 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.43640026122631315) > weight (0.25) 2021-02-20T18:38:29.192 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T18:38:29.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T18:38:29.193 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T18:38:29.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T18:38:29.188+0000 7f73a2209700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 55692) UID: 1000 2021-02-20T18:38:29.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:38:31.194 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T18:38:31.194 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:38:31.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:38:31.195 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T18:38:31.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T18:38:31.196 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T18:38:31.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T18:38:31.199 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T18:39:02.096 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9531162593459541) > weight (0.25) 2021-02-20T18:39:05.561 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4397717400741308) > weight (0.25) 2021-02-20T18:39:05.561 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T18:39:05.562 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T18:39:05.563 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T18:39:05.564 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T18:39:05.560+0000 7f2b18dbf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 57279) UID: 1000 2021-02-20T18:39:05.764 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:39:11.564 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T18:39:11.564 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:39:11.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:39:11.565 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T18:39:11.565 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T18:39:11.566 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T18:39:11.568 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T18:39:11.569 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T18:40:06.287 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T18:40:06.282+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6bab4dd40 2021-02-20T18:40:06.288 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T18:40:06.282+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ca080a40 2021-02-20T18:40:16.200 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.27266754997424214) > weight (0.25) 2021-02-20T18:40:16.208 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7978636343985822) > weight (0.25) 2021-02-20T18:40:16.208 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7764442736283982) > weight (0.25) 2021-02-20T18:40:16.208 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8032619402705511) > weight (0.25) 2021-02-20T18:40:16.208 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T18:41:09.570 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.35047393335336785) > weight (0.25) 2021-02-20T18:41:09.571 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5109146024252502) > weight (0.25) 2021-02-20T18:41:09.571 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.789536107740497) > weight (0.25) 2021-02-20T18:41:09.571 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T18:41:09.572 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T18:41:09.572 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T18:41:09.573 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T18:41:09.568+0000 7fa0ee6cb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 57184) UID: 1000 2021-02-20T18:41:09.774 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:41:13.574 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T18:41:13.574 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:41:13.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:41:13.575 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T18:41:13.575 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T18:41:13.576 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T18:41:13.579 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T18:41:13.579 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T18:41:59.210 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7286943246907545) > weight (0.25) 2021-02-20T18:41:59.210 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2533681517850468) > weight (0.25) 2021-02-20T18:41:59.210 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5220441974895783) > weight (0.25) 2021-02-20T18:41:59.211 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T18:41:59.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T18:41:59.212 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T18:41:59.212 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T18:41:59.212 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:41:59.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T18:41:59.207+0000 7fb163dfa700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 57461) UID: 1000 2021-02-20T18:41:59.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:42:05.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:42:05.314 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T18:42:05.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T18:42:05.315 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T18:42:05.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T18:42:05.318 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T18:42:41.579 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7409180194294671) > weight (0.25) 2021-02-20T18:42:41.586 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5331808999347099) > weight (0.25) 2021-02-20T18:42:41.587 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.25681484277113276) > weight (0.25) 2021-02-20T18:42:41.587 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T18:42:41.588 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T18:42:41.588 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T18:42:41.589 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T18:42:41.586+0000 7f1ff3074700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 57654) UID: 1000 2021-02-20T18:42:41.789 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:42:42.589 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T18:42:42.589 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:42:42.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:42:42.590 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T18:42:42.591 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T18:42:42.591 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T18:42:42.593 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T18:42:42.594 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T18:43:54.319 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9791248966848178) > weight (0.25) 2021-02-20T18:43:54.328 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3529975572655609) > weight (0.25) 2021-02-20T18:43:54.328 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4293445632163251) > weight (0.25) 2021-02-20T18:43:54.328 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7006680192079402) > weight (0.25) 2021-02-20T18:43:54.329 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T18:44:37.595 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8970624532052941) > weight (0.25) 2021-02-20T18:44:37.605 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4236907374739969) > weight (0.25) 2021-02-20T18:44:37.605 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.38083226464843345) > weight (0.25) 2021-02-20T18:44:37.606 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7379607257316018) > weight (0.25) 2021-02-20T18:44:37.606 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T18:45:31.330 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4895661574699557) > weight (0.25) 2021-02-20T18:45:31.337 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.59022019996189) > weight (0.25) 2021-02-20T18:45:31.338 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T18:45:31.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T18:45:31.339 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T18:45:31.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T18:45:31.337+0000 7fb24b172700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 57093) UID: 1000 2021-02-20T18:45:31.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:45:35.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.633+0000 7fe20e7cf700 -1 rbd::mirror::ImageReplayer: 0x5570f2a9ef00 [3/172baefb-5ce5-45e0-997e-249560de6fbd] start: already running 2021-02-20T18:45:35.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.633+0000 7fe20e7cf700 -1 rbd::mirror::ImageReplayer: 0x5570f2a9f180 [3/1a170226-518b-4a75-85a8-6fbdc9b5d9f0] start: already running 2021-02-20T18:45:35.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.633+0000 7fe20e7cf700 -1 rbd::mirror::ImageReplayer: 0x5570f2a9f400 [3/205dce0d-bc7c-41b6-8ed9-87e6a63d1a70] start: already running 2021-02-20T18:45:35.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.633+0000 7fe20e7cf700 -1 rbd::mirror::ImageReplayer: 0x5570f2a9f680 [3/2464eeeb-6954-426d-a8e5-129943cea55c] start: already running 2021-02-20T18:45:35.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.633+0000 7fe20e7cf700 -1 rbd::mirror::ImageReplayer: 0x5570f2a9f900 [3/3ab5666f-763c-442d-bc17-ea037dcc3a42] start: already running 2021-02-20T18:45:35.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.637+0000 7fe20e7cf700 -1 rbd::mirror::ImageReplayer: 0x5570f2a9fb80 [3/3c6940ee-a642-422e-aafb-e3920bba87cf] start: already running 2021-02-20T18:45:35.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.637+0000 7fe20e7cf700 -1 rbd::mirror::ImageReplayer: 0x5570f2a9fe00 [3/3d38caf8-6420-4f62-95dc-628caa5144da] start: already running 2021-02-20T18:45:35.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.637+0000 7fe20e7cf700 -1 rbd::mirror::ImageReplayer: 0x5570f2aa0080 [3/49a0b5eb-7d69-4824-806b-1ece12354eb9] start: already running 2021-02-20T18:45:35.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.637+0000 7fe20e7cf700 -1 rbd::mirror::ImageReplayer: 0x5570f2aa0300 [3/4c862af6-d1da-4215-b6b8-461f3883b104] start: already running 2021-02-20T18:45:35.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.637+0000 7fe20e7cf700 -1 rbd::mirror::ImageReplayer: 0x5570f2aa0580 [3/4cba354e-99d2-4859-a4d0-baeec2203656] start: already running 2021-02-20T18:45:35.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.637+0000 7fe20e7cf700 -1 rbd::mirror::ImageReplayer: 0x5570f2aa0800 [3/567ace7d-44e4-4d42-b47a-38bd35ec0e11] start: already running 2021-02-20T18:45:35.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20efd0700 -1 rbd::mirror::ImageReplayer: 0x5570f6b5ac80 [3/941323f6-0453-4944-ab79-664fde6ec580] start: already running 2021-02-20T18:45:35.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20efd0700 -1 rbd::mirror::ImageReplayer: 0x5570f6b5af00 [3/945695ad-0f15-4fd7-a092-4e7df53e1d1b] start: already running 2021-02-20T18:45:35.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20efd0700 -1 rbd::mirror::ImageReplayer: 0x5570f2a9ef00 [3/172baefb-5ce5-45e0-997e-249560de6fbd] start: already running 2021-02-20T18:45:35.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20efd0700 -1 rbd::mirror::ImageReplayer: 0x5570f2a9f180 [3/1a170226-518b-4a75-85a8-6fbdc9b5d9f0] start: already running 2021-02-20T18:45:35.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20efd0700 -1 rbd::mirror::ImageReplayer: 0x5570f2a9f400 [3/205dce0d-bc7c-41b6-8ed9-87e6a63d1a70] start: already running 2021-02-20T18:45:35.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20efd0700 -1 rbd::mirror::ImageReplayer: 0x5570f2a9f680 [3/2464eeeb-6954-426d-a8e5-129943cea55c] start: already running 2021-02-20T18:45:35.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20efd0700 -1 rbd::mirror::ImageReplayer: 0x5570f2a9f900 [3/3ab5666f-763c-442d-bc17-ea037dcc3a42] start: already running 2021-02-20T18:45:35.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20efd0700 -1 rbd::mirror::ImageReplayer: 0x5570f2a9fb80 [3/3c6940ee-a642-422e-aafb-e3920bba87cf] start: already running 2021-02-20T18:45:35.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20efd0700 -1 rbd::mirror::ImageReplayer: 0x5570f2a9fe00 [3/3d38caf8-6420-4f62-95dc-628caa5144da] start: already running 2021-02-20T18:45:35.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20e7cf700 -1 rbd::mirror::ImageReplayer: 0x5570f2aa0300 [3/4c862af6-d1da-4215-b6b8-461f3883b104] start: already running 2021-02-20T18:45:35.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20e7cf700 -1 rbd::mirror::ImageReplayer: 0x5570f2aa0580 [3/4cba354e-99d2-4859-a4d0-baeec2203656] start: already running 2021-02-20T18:45:35.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20e7cf700 -1 rbd::mirror::ImageReplayer: 0x5570f2aa0800 [3/567ace7d-44e4-4d42-b47a-38bd35ec0e11] start: already running 2021-02-20T18:45:35.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20e7cf700 -1 rbd::mirror::ImageReplayer: 0x5570f2aa1480 [3/5786ec48-0da5-470d-b6a6-ddd71a666657] start: already running 2021-02-20T18:45:35.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20efd0700 -1 rbd::mirror::ImageReplayer: 0x5570f2aa1700 [3/5c036645-b242-4fa8-bde0-d5b28dda3fd3] start: already running 2021-02-20T18:45:35.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20efd0700 -1 rbd::mirror::ImageReplayer: 0x5570f2aa1980 [3/5f3204b5-90c6-4244-a940-e749860457d5] start: already running 2021-02-20T18:45:35.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20efd0700 -1 rbd::mirror::ImageReplayer: 0x5570f2aa0a80 [3/675a1b7b-4e17-4b1e-9e92-eaf406f1c33f] start: already running 2021-02-20T18:45:35.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20e7cf700 -1 rbd::mirror::ImageReplayer: 0x5570f2aa0d00 [3/681dbea5-7557-4dba-a92d-01da85613e31] start: already running 2021-02-20T18:45:35.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20e7cf700 -1 rbd::mirror::ImageReplayer: 0x5570f2aa0f80 [3/753c14e7-97d4-4109-8814-2c7f1abbbee8] start: already running 2021-02-20T18:45:35.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20e7cf700 -1 rbd::mirror::ImageReplayer: 0x5570f2aa1200 [3/7658e9e0-5116-4c47-bad5-4e7894e27da7] start: already running 2021-02-20T18:45:35.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20efd0700 -1 rbd::mirror::ImageReplayer: 0x5570f2aa1c00 [3/790c038c-4353-4ff1-ae0b-9c9bb1e1adaa] start: already running 2021-02-20T18:45:35.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20efd0700 -1 rbd::mirror::ImageReplayer: 0x5570f6b5a000 [3/7fb27511-6f76-444f-b338-191b2c7caa61] start: already running 2021-02-20T18:45:35.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20e7cf700 -1 rbd::mirror::ImageReplayer: 0x5570f6b5a280 [3/858c7e91-92b9-4e8b-8ae9-f8cb46e242bf] start: already running 2021-02-20T18:45:35.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20e7cf700 -1 rbd::mirror::ImageReplayer: 0x5570f6b5a500 [3/87eed4f8-1d97-46e0-b439-8f6b85b99b40] start: already running 2021-02-20T18:45:35.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20efd0700 -1 rbd::mirror::ImageReplayer: 0x5570f6b5a780 [3/8c216315-5117-4846-b850-c4265b4697a3] start: already running 2021-02-20T18:45:35.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:35.669+0000 7fe20efd0700 -1 rbd::mirror::ImageReplayer: 0x5570f6b5aa00 [3/91672d77-7c82-4d00-b945-306681ce602c] start: already running 2021-02-20T18:45:36.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:45:36.125+0000 7fe20efd0700 -1 rbd::mirror::ImageReplayer: 0x5570f2aa0f80 [3/753c14e7-97d4-4109-8814-2c7f1abbbee8] register_admin_socket_hook: error registering admin socket commands 2021-02-20T18:45:39.340 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T18:45:39.340 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:45:39.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:45:39.341 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T18:45:39.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T18:45:39.342 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T18:45:39.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T18:45:39.346 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T18:45:55.606 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3950005662233227) > weight (0.25) 2021-02-20T18:45:55.617 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8502903162972905) > weight (0.25) 2021-02-20T18:45:55.617 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6135097828164188) > weight (0.25) 2021-02-20T18:45:55.617 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T18:45:55.618 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T18:45:55.618 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T18:45:55.618 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T18:45:55.619 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:45:55.619 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T18:45:55.616+0000 7fd7622d0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 57837) UID: 1000 2021-02-20T18:45:55.819 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:46:01.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:46:01.728 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T18:46:01.728 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T18:46:01.729 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T18:46:01.732 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T18:46:01.732 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T18:47:08.347 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:47:08.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:47:08.354 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:47:08.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:47:08.351+0000 7fe20f7d1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 57370) UID: 1000 2021-02-20T18:47:08.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:47:15.355 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:47:15.356 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:47:15.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:47:15.357 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:47:15.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:47:15.357 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:47:15.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:47:15.361 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T18:47:48.733 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:47:48.734 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:47:48.734 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T18:47:48.735 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T18:47:48.729+0000 7f920e8a0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 56450) UID: 1000 2021-02-20T18:47:48.936 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:47:54.736 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:47:54.736 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:47:54.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:47:54.737 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:47:54.737 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:47:54.738 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:47:54.740 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:47:54.741 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T18:49:09.362 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:49:09.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:49:09.364 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T18:49:09.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:49:09.359+0000 7f38df306700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 58131) UID: 1000 2021-02-20T18:49:09.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:49:15.365 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:49:15.365 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:49:15.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:49:15.366 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:49:15.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:49:15.367 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:49:15.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:49:15.370 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T18:49:52.742 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:49:52.743 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:49:52.743 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T18:49:52.744 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:49:52.744 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:49:52.745 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T18:49:52.738+0000 7f7f4599a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 58222) UID: 1000 2021-02-20T18:49:52.945 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:49:58.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:49:58.846 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:49:58.847 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:49:58.847 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:49:58.850 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:49:58.850 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T18:51:11.371 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:51:11.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:51:11.373 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:51:11.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:51:11.368+0000 7f1169e89700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 58313) UID: 1000 2021-02-20T18:51:11.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:51:18.374 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:51:18.374 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:51:18.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:51:18.376 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:51:18.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:51:18.377 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:51:18.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:51:18.381 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T18:51:40.851 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:51:40.852 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:51:40.852 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T18:51:40.853 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T18:51:40.847+0000 7fe227a22700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 58405) UID: 1000 2021-02-20T18:51:41.053 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:51:49.853 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:51:49.855 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:51:49.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:51:49.856 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:51:49.856 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:51:49.856 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:51:49.860 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:51:49.860 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T18:52:33.382 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:52:33.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:52:33.397 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T18:52:33.397 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:52:33.398 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:52:33.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T18:52:33.390+0000 7f058cb29700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 58497) UID: 1000 2021-02-20T18:52:33.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:52:39.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:52:39.501 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:52:39.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:52:39.502 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:52:39.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:52:39.507 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T18:53:16.862 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5723794506327821) > weight (0.25) 2021-02-20T18:53:16.870 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8772117647685131) > weight (0.25) 2021-02-20T18:53:16.871 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6412689458590743) > weight (0.25) 2021-02-20T18:53:16.871 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7857738106060687) > weight (0.25) 2021-02-20T18:53:16.871 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T18:54:26.508 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6455882950977954) > weight (0.25) 2021-02-20T18:54:26.516 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6705183535693608) > weight (0.25) 2021-02-20T18:54:26.516 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.48611879733441576) > weight (0.25) 2021-02-20T18:54:26.517 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6573508475007062) > weight (0.25) 2021-02-20T18:54:26.517 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T18:54:47.872 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7152143386282213) > weight (0.25) 2021-02-20T18:54:47.872 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.36131918624974035) > weight (0.25) 2021-02-20T18:54:47.873 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7923627898463684) > weight (0.25) 2021-02-20T18:54:47.873 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T18:54:47.874 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T18:54:47.874 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T18:54:47.875 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T18:54:47.866+0000 7f3e07b9b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 58031) UID: 1000 2021-02-20T18:54:48.075 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:54:49.875 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T18:54:49.875 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:54:49.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:54:49.876 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T18:54:49.877 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T18:54:49.877 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T18:54:49.880 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T18:54:49.880 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T18:56:04.518 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.28558136086999397) > weight (0.25) 2021-02-20T18:56:04.527 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9764476227200498) > weight (0.25) 2021-02-20T18:56:04.527 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9063377636202488) > weight (0.25) 2021-02-20T18:56:04.527 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.39617038428902374) > weight (0.25) 2021-02-20T18:56:04.527 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T18:56:04.880 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2649917097523159) > weight (0.25) 2021-02-20T18:56:04.881 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T18:56:04.882 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T18:56:04.882 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:56:04.883 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T18:56:04.880+0000 7fd99ff38700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 56727) UID: 1000 2021-02-20T18:56:05.284 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:56:11.884 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T18:56:11.885 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:56:11.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:56:11.886 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T18:56:11.886 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T18:56:11.887 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T18:56:11.891 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T18:56:11.892 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T18:57:17.529 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5041894981702788) > weight (0.25) 2021-02-20T18:57:17.535 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6761280046329704) > weight (0.25) 2021-02-20T18:57:17.536 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5299371307605487) > weight (0.25) 2021-02-20T18:57:17.536 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6996952073355445) > weight (0.25) 2021-02-20T18:57:17.536 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T18:57:48.893 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7824683763913896) > weight (0.25) 2021-02-20T18:57:48.893 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8814681360768204) > weight (0.25) 2021-02-20T18:57:48.893 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7935650724173218) > weight (0.25) 2021-02-20T18:57:48.893 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T18:57:48.894 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T18:57:48.895 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T18:57:48.896 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T18:57:48.889+0000 7fc00ccea700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 58772) UID: 1000 2021-02-20T18:57:49.096 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:57:51.896 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T18:57:51.896 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:57:51.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:57:51.897 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T18:57:51.897 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T18:57:51.898 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T18:57:51.901 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T18:57:51.901 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T18:58:49.537 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.48012002355851835) > weight (0.25) 2021-02-20T18:58:49.545 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.32678549988926087) > weight (0.25) 2021-02-20T18:58:49.545 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6519446343238406) > weight (0.25) 2021-02-20T18:58:49.546 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T18:58:49.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T18:58:49.547 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T18:58:49.547 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T18:58:49.547 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:58:49.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T18:58:49.544+0000 7fcdbc7cc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 57746) UID: 1000 2021-02-20T18:58:49.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:58:55.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:59:12.974 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T18:59:12.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T18:59:12.975 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T18:59:12.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T18:59:12.978 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T18:59:41.902 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8930630151055412) > weight (0.25) 2021-02-20T18:59:41.903 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5506608722353142) > weight (0.25) 2021-02-20T18:59:41.903 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8776624567333239) > weight (0.25) 2021-02-20T18:59:41.903 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9143457087930515) > weight (0.25) 2021-02-20T18:59:41.904 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T19:00:20.980 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8696888198549421) > weight (0.25) 2021-02-20T19:00:20.987 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8677762183004069) > weight (0.25) 2021-02-20T19:00:20.987 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T19:00:20.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T19:00:20.988 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T19:00:20.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T19:00:20.985+0000 7ff34945c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 57940) UID: 1000 2021-02-20T19:00:21.232 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T19:00:21.222+0000 7fb8259e4700 -1 reset not still connected to 0x5578525087d0 2021-02-20T19:00:21.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:00:22.990 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T19:00:22.990 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:00:22.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:00:22.991 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T19:00:22.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T19:00:22.992 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T19:00:22.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T19:00:22.994 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T19:00:44.904 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5694840246140968) > weight (0.25) 2021-02-20T19:00:44.905 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T19:00:44.906 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T19:00:44.907 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T19:00:44.907 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T19:00:44.908 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:00:44.909 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T19:00:44.904+0000 7fc303b02700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 58867) UID: 1000 2021-02-20T19:00:45.108 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:00:51.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:00:54.716 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T19:00:54.725 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T19:00:54.726 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T19:00:54.729 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T19:00:54.730 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T19:01:35.996 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2573510524181051) > weight (0.25) 2021-02-20T19:01:36.005 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5137001436995047) > weight (0.25) 2021-02-20T19:01:36.005 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4731687528335874) > weight (0.25) 2021-02-20T19:01:36.005 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7535860205790189) > weight (0.25) 2021-02-20T19:01:36.006 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T19:02:28.279 INFO:tasks.ceph.cluster1.osd.1.smithi170.stderr:2021-02-20T19:02:28.267+0000 7fb102cd5700 -1 reset not still connected to 0x559d5261e5b0 2021-02-20T19:02:28.285 INFO:tasks.ceph.cluster1.osd.1.smithi170.stderr:2021-02-20T19:02:28.271+0000 7fb102cd5700 -1 reset not still connected to 0x559d534dcd80 2021-02-20T19:02:28.285 INFO:tasks.ceph.cluster1.osd.1.smithi170.stderr:2021-02-20T19:02:28.271+0000 7fb102cd5700 -1 reset not still connected to 0x559d60490220 2021-02-20T19:02:53.731 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7838304139309171) > weight (0.25) 2021-02-20T19:03:00.414 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.38831596179081374) > weight (0.25) 2021-02-20T19:03:00.415 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.25914506166249196) > weight (0.25) 2021-02-20T19:03:00.415 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T19:03:00.416 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T19:03:00.416 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T19:03:00.417 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T19:03:00.413+0000 7f356585a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 58959) UID: 1000 2021-02-20T19:03:00.618 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:03:01.418 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T19:03:01.418 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:03:01.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:03:01.419 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T19:03:01.419 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T19:03:01.420 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T19:03:01.422 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T19:03:01.423 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T19:03:18.006 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8287756828708627) > weight (0.25) 2021-02-20T19:03:21.302 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:03:21.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:03:21.303 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T19:03:21.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T19:03:21.300+0000 7f21e0946700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 55981) UID: 1000 2021-02-20T19:03:21.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:03:23.304 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:03:23.305 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:03:23.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:03:23.305 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:03:23.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:03:23.306 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:03:23.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:03:23.309 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T19:04:39.424 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8987302314763244) > weight (0.25) 2021-02-20T19:04:39.432 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T19:04:39.432 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T19:04:39.433 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T19:04:39.434 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T19:04:39.426+0000 7fdccf253700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 59242) UID: 1000 2021-02-20T19:04:39.634 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:04:43.434 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T19:04:43.434 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:04:43.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:04:43.435 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T19:04:43.436 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T19:04:43.436 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T19:04:43.439 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T19:04:43.439 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T19:05:19.310 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:05:19.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:05:19.321 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T19:05:19.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T19:05:19.317+0000 7ffa2a339700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 58681) UID: 1000 2021-02-20T19:05:19.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:05:21.321 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:05:21.322 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:05:21.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:05:21.323 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:05:21.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:05:21.323 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:05:21.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:05:21.328 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T19:06:03.440 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5368062291344781) > weight (0.25) 2021-02-20T19:06:03.448 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.45688817308112895) > weight (0.25) 2021-02-20T19:06:03.448 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5004169767713229) > weight (0.25) 2021-02-20T19:06:03.448 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8616158134649659) > weight (0.25) 2021-02-20T19:06:03.449 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T19:06:23.329 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:06:36.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:06:36.889 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T19:06:36.889 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:06:36.889 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:06:36.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T19:06:36.875+0000 7f53f48c6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 59611) UID: 1000 2021-02-20T19:06:37.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:06:42.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:06:42.992 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:06:42.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:06:42.993 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:06:42.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:06:42.997 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T19:07:10.450 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:07:10.457 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:07:10.457 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T19:07:10.458 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T19:07:10.450+0000 7f5d4dfe8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 58589) UID: 1000 2021-02-20T19:07:10.659 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:07:12.459 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:07:12.459 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:07:12.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:07:12.460 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:07:12.460 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:07:12.461 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:07:12.463 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:07:12.464 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T19:08:14.999 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:08:15.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:08:15.007 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T19:08:15.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T19:08:15.004+0000 7f09dc3b6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 59705) UID: 1000 2021-02-20T19:08:15.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:08:19.008 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:08:19.014 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:08:19.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:08:19.015 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:08:19.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:08:19.016 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:08:19.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:08:19.020 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T19:09:01.465 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8272076899570228) > weight (0.25) 2021-02-20T19:09:01.472 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.802620545045792) > weight (0.25) 2021-02-20T19:09:01.473 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6142461973785058) > weight (0.25) 2021-02-20T19:09:01.473 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T19:09:01.474 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T19:09:01.474 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T19:09:01.475 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T19:09:01.467+0000 7f7924e68700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 59335) UID: 1000 2021-02-20T19:09:01.675 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:09:10.475 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T19:09:10.476 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:09:10.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:09:10.477 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T19:09:10.477 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T19:09:10.477 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T19:09:10.480 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T19:09:10.480 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T19:09:45.021 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5511590418165546) > weight (0.25) 2021-02-20T19:09:45.206 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8074576262945866) > weight (0.25) 2021-02-20T19:09:45.207 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9956139222400526) > weight (0.25) 2021-02-20T19:09:45.207 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8507401656839619) > weight (0.25) 2021-02-20T19:09:45.208 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T19:10:10.482 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6553181322040849) > weight (0.25) 2021-02-20T19:10:10.488 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4769116687612811) > weight (0.25) 2021-02-20T19:10:10.489 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6817767152819674) > weight (0.25) 2021-02-20T19:10:10.489 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9980673772252321) > weight (0.25) 2021-02-20T19:10:10.489 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T19:11:35.209 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.33051833036829026) > weight (0.25) 2021-02-20T19:11:35.217 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:11:35.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:11:35.218 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T19:11:35.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T19:11:35.215+0000 7f2910970700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 59426) UID: 1000 2021-02-20T19:11:35.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:11:41.220 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:11:41.220 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:11:41.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:11:41.221 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:11:41.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:11:41.222 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:11:41.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:11:41.225 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T19:12:05.490 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:12:05.498 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:12:05.498 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T19:12:05.499 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T19:12:05.490+0000 7f533ea24700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 59797) UID: 1000 2021-02-20T19:12:05.700 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:12:14.500 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:12:14.509 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:12:14.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:12:14.510 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:12:14.510 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:12:14.511 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:12:14.514 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:12:14.514 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T19:13:13.226 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3730063222341543) > weight (0.25) 2021-02-20T19:13:13.227 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:13:13.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:13:13.228 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T19:13:13.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T19:13:13.224+0000 7f4748a29700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 60072) UID: 1000 2021-02-20T19:13:13.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:13:15.515 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.885895042303207) > weight (0.25) 2021-02-20T19:13:16.606 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9120273701854573) > weight (0.25) 2021-02-20T19:13:16.607 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9423427403447903) > weight (0.25) 2021-02-20T19:13:16.607 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.26627756219048093) > weight (0.25) 2021-02-20T19:13:16.607 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T19:13:17.229 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:13:17.229 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:13:17.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:13:17.230 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:13:17.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:13:17.230 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:13:17.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:13:17.234 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T19:14:31.609 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8385619230225344) > weight (0.25) 2021-02-20T19:14:31.609 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T19:14:31.610 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T19:14:31.611 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T19:14:31.611 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T19:14:31.606+0000 7ff86d7fb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 59517) UID: 1000 2021-02-20T19:14:31.812 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:14:35.612 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T19:14:35.612 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:14:35.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:14:35.613 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T19:14:35.613 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T19:14:35.613 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T19:14:35.616 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T19:14:35.616 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T19:14:44.235 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6800286886721622) > weight (0.25) 2021-02-20T19:14:44.241 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6746811298225122) > weight (0.25) 2021-02-20T19:14:44.241 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9956452633949396) > weight (0.25) 2021-02-20T19:14:44.242 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.306839621500129) > weight (0.25) 2021-02-20T19:14:44.242 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T19:15:43.907 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T19:15:43.900+0000 7fb8259e4700 -1 reset not still connected to 0x5578525087d0 2021-02-20T19:15:43.908 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T19:15:43.900+0000 7fb8259e4700 -1 reset not still connected to 0x557855c1d1e0 2021-02-20T19:15:43.987 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T19:15:43.900+0000 7fb8259e4700 -1 reset not still connected to 0x557855c1d860 2021-02-20T19:15:43.987 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T19:15:43.900+0000 7fb8259e4700 -1 reset not still connected to 0x557855c1e630 2021-02-20T19:15:43.988 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T19:15:43.900+0000 7fb8259e4700 -1 reset not still connected to 0x557863950340 2021-02-20T19:15:43.988 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T19:15:43.900+0000 7fb8259e4700 -1 reset not still connected to 0x5578639509c0 2021-02-20T19:15:51.243 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8561714004193246) > weight (0.25) 2021-02-20T19:15:51.243 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9869325941357007) > weight (0.25) 2021-02-20T19:15:51.244 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6892724793420062) > weight (0.25) 2021-02-20T19:15:51.244 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.26562090208226274) > weight (0.25) 2021-02-20T19:15:51.245 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T19:15:54.618 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:15:54.723 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:15:54.724 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T19:15:54.724 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T19:15:54.720+0000 7fc969294700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 60166) UID: 1000 2021-02-20T19:15:54.925 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:15:58.724 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:15:58.736 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:15:58.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:15:58.737 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:15:58.737 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:15:58.752 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:15:58.757 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:15:58.758 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T19:17:04.759 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2687589184644228) > weight (0.25) 2021-02-20T19:17:05.274 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9286468035991784) > weight (0.25) 2021-02-20T19:17:05.275 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7984968053357163) > weight (0.25) 2021-02-20T19:17:05.275 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5513264467779957) > weight (0.25) 2021-02-20T19:17:05.275 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T19:17:13.192 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T19:17:13.182+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6c083ba00 2021-02-20T19:17:13.193 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T19:17:13.182+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6c083bc70 2021-02-20T19:17:13.193 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T19:17:13.182+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6c083c7d0 2021-02-20T19:17:13.193 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T19:17:13.182+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ca07f2b0 2021-02-20T19:17:13.194 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T19:17:13.182+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ca07f930 2021-02-20T19:17:48.246 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:17:48.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:17:48.253 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T19:17:48.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T19:17:48.249+0000 7f8020ab3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 59888) UID: 1000 2021-02-20T19:17:48.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:17:54.254 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:17:54.255 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:17:54.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:17:54.256 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:17:54.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:17:54.257 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:17:54.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:17:54.260 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T19:18:44.276 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:18:44.283 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:18:44.283 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T19:18:44.284 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T19:18:44.279+0000 7feb1fadc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 60442) UID: 1000 2021-02-20T19:18:44.485 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:18:47.284 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:18:47.291 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:18:47.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:18:47.293 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:18:47.293 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:18:47.293 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:18:47.297 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:18:47.298 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T19:19:02.260 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5920360462218426) > weight (0.25) 2021-02-20T19:19:02.261 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:19:02.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:19:02.262 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T19:19:02.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T19:19:02.255+0000 7fedea8b3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 60257) UID: 1000 2021-02-20T19:19:02.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:19:04.263 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:19:04.263 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:19:04.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:19:04.264 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:19:04.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:19:04.265 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:19:04.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:19:04.268 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T19:19:56.299 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.38953769703136143) > weight (0.25) 2021-02-20T19:19:56.308 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.32564952826102556) > weight (0.25) 2021-02-20T19:19:56.309 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9904289960678497) > weight (0.25) 2021-02-20T19:19:56.309 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8312151359574276) > weight (0.25) 2021-02-20T19:19:56.309 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T19:20:22.268 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4259337404420558) > weight (0.25) 2021-02-20T19:20:22.277 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8600478181318588) > weight (0.25) 2021-02-20T19:20:22.278 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.41544262488625117) > weight (0.25) 2021-02-20T19:20:22.278 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4996958726738183) > weight (0.25) 2021-02-20T19:20:22.278 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T19:21:28.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5524119344634857) > weight (0.25) 2021-02-20T19:21:28.289 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8133432513084965) > weight (0.25) 2021-02-20T19:21:28.289 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6692240803810271) > weight (0.25) 2021-02-20T19:21:28.289 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5090852237037119) > weight (0.25) 2021-02-20T19:21:28.290 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T19:21:31.311 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5842410747040476) > weight (0.25) 2021-02-20T19:21:32.663 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.39008896446886565) > weight (0.25) 2021-02-20T19:21:32.663 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T19:21:32.664 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T19:21:32.665 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T19:21:32.666 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T19:21:32.659+0000 7f01261d1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 57561) UID: 1000 2021-02-20T19:21:32.867 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:21:39.665 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T19:21:39.666 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:21:39.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:21:39.667 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T19:21:39.668 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T19:21:39.668 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T19:21:39.671 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T19:21:39.671 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T19:22:40.757 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T19:22:40.749+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b6c88970 2021-02-20T19:22:40.760 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T19:22:40.749+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6c39267d0 2021-02-20T19:22:40.760 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T19:22:40.749+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6e58ac080 2021-02-20T19:22:59.291 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7195831900426329) > weight (0.25) 2021-02-20T19:23:02.669 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9628198399385136) > weight (0.25) 2021-02-20T19:23:02.669 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7092243869445873) > weight (0.25) 2021-02-20T19:23:02.669 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T19:23:02.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T19:23:02.672 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T19:23:02.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T19:23:02.665+0000 7f3328d82700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 59051) UID: 1000 2021-02-20T19:23:03.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:23:09.673 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T19:23:09.674 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:23:09.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:23:09.674 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T19:23:09.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T19:23:09.675 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T19:23:09.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T19:23:09.678 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T19:23:10.857 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T19:23:10.777+0000 7fb8259e4700 -1 reset not still connected to 0x55785c651a80 2021-02-20T19:23:27.672 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5112175373665482) > weight (0.25) 2021-02-20T19:23:27.672 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.767576984957982) > weight (0.25) 2021-02-20T19:23:27.673 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T19:23:27.673 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T19:23:27.673 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T19:23:27.674 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T19:23:27.674 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:23:27.675 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T19:23:27.668+0000 7fb84f396700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 60813) UID: 1000 2021-02-20T19:23:27.875 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:23:33.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:23:33.776 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T19:23:33.777 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T19:23:33.777 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T19:23:33.780 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T19:23:33.780 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T19:24:57.679 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.992598971412533) > weight (0.25) 2021-02-20T19:24:57.687 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5089771152874559) > weight (0.25) 2021-02-20T19:24:57.687 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4428008871953736) > weight (0.25) 2021-02-20T19:24:57.687 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.34830754942022957) > weight (0.25) 2021-02-20T19:24:57.688 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T19:25:04.781 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5349049631961857) > weight (0.25) 2021-02-20T19:25:04.790 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.783607165712849) > weight (0.25) 2021-02-20T19:25:04.791 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T19:25:04.791 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T19:25:04.792 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T19:25:04.792 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T19:25:04.792 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:25:04.793 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T19:25:04.785+0000 7f5243334700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 61005) UID: 1000 2021-02-20T19:25:04.993 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:25:10.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:25:10.895 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T19:25:10.895 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T19:25:10.896 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T19:25:10.898 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T19:25:10.899 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T19:26:42.689 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6111483718857497) > weight (0.25) 2021-02-20T19:26:44.177 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5170938004210986) > weight (0.25) 2021-02-20T19:26:44.178 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T19:26:44.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T19:26:44.179 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T19:26:44.179 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T19:26:44.179 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:26:44.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T19:26:44.171+0000 7fb7adbe8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 59143) UID: 1000 2021-02-20T19:26:44.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:26:50.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:26:50.281 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T19:26:50.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T19:26:50.282 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T19:26:50.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T19:26:50.285 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T19:27:02.899 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4082431313670718) > weight (0.25) 2021-02-20T19:27:02.900 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7453411412527385) > weight (0.25) 2021-02-20T19:27:02.900 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9555611424250119) > weight (0.25) 2021-02-20T19:27:02.900 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4534336417827324) > weight (0.25) 2021-02-20T19:27:02.901 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T19:28:07.902 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7750245118983776) > weight (0.25) 2021-02-20T19:28:07.911 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T19:28:07.912 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T19:28:07.913 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T19:28:07.913 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T19:28:07.913 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:28:07.915 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T19:28:07.908+0000 7f7e6e942700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 60348) UID: 1000 2021-02-20T19:28:08.113 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:28:14.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:28:14.016 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T19:28:14.016 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T19:28:14.017 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T19:28:14.020 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T19:28:14.020 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T19:28:49.286 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8439538938764624) > weight (0.25) 2021-02-20T19:28:49.294 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9805414181521012) > weight (0.25) 2021-02-20T19:28:49.294 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7109735993656136) > weight (0.25) 2021-02-20T19:28:49.295 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.26442581926082154) > weight (0.25) 2021-02-20T19:28:49.295 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T19:29:37.021 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5105758145974036) > weight (0.25) 2021-02-20T19:29:37.031 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9971439936979282) > weight (0.25) 2021-02-20T19:29:37.031 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T19:29:37.032 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T19:29:37.032 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T19:29:37.034 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T19:29:37.030+0000 7feb15410700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 61099) UID: 1000 2021-02-20T19:29:37.234 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:29:40.033 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T19:29:40.034 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:29:40.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:29:40.035 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T19:29:40.035 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T19:29:40.035 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T19:29:40.038 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T19:29:40.038 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T19:30:05.296 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:30:05.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:30:05.297 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T19:30:05.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T19:30:05.289+0000 7fd5caa05700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 60539) UID: 1000 2021-02-20T19:30:05.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:30:09.299 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:30:09.299 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:30:09.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:30:09.300 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:30:09.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:30:09.301 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:30:09.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:30:09.304 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T19:30:41.040 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:30:41.040 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:30:41.041 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T19:30:41.042 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T19:30:41.036+0000 7f37e98e1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 60630) UID: 1000 2021-02-20T19:30:41.242 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:30:44.042 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:30:44.043 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:30:44.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:30:44.043 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:30:44.044 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:30:44.044 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:30:44.047 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:30:44.047 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T19:31:14.305 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:31:14.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:31:14.318 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T19:31:14.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T19:31:14.311+0000 7fe5d659f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 61468) UID: 1000 2021-02-20T19:31:14.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:31:15.319 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:31:15.319 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:31:15.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:31:15.320 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:31:15.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:31:15.321 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:31:15.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:31:15.323 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T19:32:07.048 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:32:07.057 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:32:07.057 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T19:32:07.058 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T19:32:07.050+0000 7f38cfb26700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 61559) UID: 1000 2021-02-20T19:32:07.260 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:32:08.058 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:32:08.059 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:32:08.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:32:08.059 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:32:08.060 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:32:08.060 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:32:08.067 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:32:08.068 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T19:33:01.324 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4191044522559392) > weight (0.25) 2021-02-20T19:33:01.332 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:33:01.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:33:01.333 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T19:33:01.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T19:33:01.328+0000 7ff498020700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 60721) UID: 1000 2021-02-20T19:33:01.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:33:06.334 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:33:06.334 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:33:06.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:33:06.335 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:33:06.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:33:06.336 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:33:06.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:33:06.339 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T19:33:14.069 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7546813964867076) > weight (0.25) 2021-02-20T19:33:14.077 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.63829658409336) > weight (0.25) 2021-02-20T19:33:14.077 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T19:33:14.078 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T19:33:14.078 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T19:33:14.079 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T19:33:14.076+0000 7fbb6aa77700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 61376) UID: 1000 2021-02-20T19:33:14.279 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:33:19.079 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T19:33:19.079 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:33:19.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:33:19.080 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T19:33:19.080 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T19:33:19.080 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T19:33:19.083 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T19:33:19.084 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T19:34:24.085 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:34:24.087 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:34:24.087 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T19:34:24.088 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T19:34:24.082+0000 7f898bc32700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 61741) UID: 1000 2021-02-20T19:34:24.289 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:34:28.089 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:34:28.089 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:34:28.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:34:28.090 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:34:28.090 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:34:28.091 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:34:28.094 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:34:28.094 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T19:34:35.340 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7018925899309567) > weight (0.25) 2021-02-20T19:34:35.341 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6500263492785845) > weight (0.25) 2021-02-20T19:34:35.341 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.42776327066419384) > weight (0.25) 2021-02-20T19:34:35.341 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9543322138748793) > weight (0.25) 2021-02-20T19:34:35.342 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T19:35:43.343 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8897742941342535) > weight (0.25) 2021-02-20T19:35:43.353 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:35:43.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:35:43.356 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T19:35:43.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T19:35:43.352+0000 7f7f79fde700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 61841) UID: 1000 2021-02-20T19:35:43.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:35:50.358 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:35:50.363 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:35:50.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:35:50.364 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:35:50.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:35:50.365 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:35:50.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:35:50.367 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T19:36:08.094 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7669130716493935) > weight (0.25) 2021-02-20T19:36:08.102 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.27225923691178733) > weight (0.25) 2021-02-20T19:36:08.103 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T19:36:08.103 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T19:36:08.103 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T19:36:08.104 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T19:36:08.099+0000 7efda285c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 61932) UID: 1000 2021-02-20T19:36:08.305 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:36:17.105 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T19:36:18.333 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:36:18.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:36:18.334 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T19:36:18.334 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T19:36:18.335 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T19:36:18.338 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T19:36:18.338 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T19:37:01.369 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:37:01.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:37:01.370 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T19:37:01.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T19:37:01.366+0000 7f333da62700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 61650) UID: 1000 2021-02-20T19:37:01.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:37:02.371 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:37:02.371 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:37:02.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:37:02.372 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:37:02.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:37:02.373 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:37:02.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:37:02.376 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T19:37:26.339 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.30236478491991203) > weight (0.25) 2021-02-20T19:37:26.348 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4089086017814284) > weight (0.25) 2021-02-20T19:37:26.348 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.42287773211328583) > weight (0.25) 2021-02-20T19:37:26.348 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T19:37:26.349 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T19:37:26.350 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T19:37:26.351 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T19:37:26.345+0000 7f4163bd2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 59979) UID: 1000 2021-02-20T19:37:26.751 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:37:34.351 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T19:37:34.352 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:37:34.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:37:34.353 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T19:37:34.353 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T19:37:34.353 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T19:37:34.356 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T19:37:34.356 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T19:38:30.377 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:38:30.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:38:30.384 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T19:38:30.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T19:38:30.383+0000 7f7fff151700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 62301) UID: 1000 2021-02-20T19:38:30.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:38:31.385 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:38:31.386 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:38:31.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:38:31.387 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:38:31.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:38:31.388 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:38:31.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:38:31.393 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T19:39:10.357 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:39:10.358 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:39:10.359 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T19:39:10.359 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:39:10.359 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:39:10.360 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T19:39:10.354+0000 7fb793b82700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 62024) UID: 1000 2021-02-20T19:39:10.560 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:39:16.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:39:16.461 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:39:16.462 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:39:16.462 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:39:16.465 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:39:16.465 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T19:40:07.395 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9426660189715038) > weight (0.25) 2021-02-20T19:40:07.405 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5491486709931346) > weight (0.25) 2021-02-20T19:40:07.405 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7586915530575379) > weight (0.25) 2021-02-20T19:40:07.406 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7572030424046756) > weight (0.25) 2021-02-20T19:40:07.406 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T19:40:26.466 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.639888085911282) > weight (0.25) 2021-02-20T19:40:26.474 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T19:40:26.474 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T19:40:26.475 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T19:40:26.475 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T19:40:26.475 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:40:26.479 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T19:40:26.476+0000 7f40638aa700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 61284) UID: 1000 2021-02-20T19:40:26.679 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:40:32.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:40:32.578 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T19:40:32.578 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T19:40:32.579 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T19:40:32.581 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T19:40:32.582 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T19:41:48.583 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2553755676013011) > weight (0.25) 2021-02-20T19:41:48.593 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.28721940791651923) > weight (0.25) 2021-02-20T19:41:48.593 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.34919718381896014) > weight (0.25) 2021-02-20T19:41:48.593 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9921445697350957) > weight (0.25) 2021-02-20T19:41:48.594 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T19:41:51.407 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8702261920329075) > weight (0.25) 2021-02-20T19:41:51.407 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7552880968705931) > weight (0.25) 2021-02-20T19:41:51.407 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2770426872799179) > weight (0.25) 2021-02-20T19:41:51.408 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8046802136835889) > weight (0.25) 2021-02-20T19:41:51.408 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T19:42:54.595 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6255764633471887) > weight (0.25) 2021-02-20T19:42:54.603 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8241865393839416) > weight (0.25) 2021-02-20T19:42:54.603 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7892984567556621) > weight (0.25) 2021-02-20T19:42:54.603 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9146383096390248) > weight (0.25) 2021-02-20T19:42:54.603 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T19:42:55.409 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6993641898235218) > weight (0.25) 2021-02-20T19:42:55.409 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.26736061188873517) > weight (0.25) 2021-02-20T19:42:55.409 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7830505846759102) > weight (0.25) 2021-02-20T19:42:55.409 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.955323168069624) > weight (0.25) 2021-02-20T19:42:55.410 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T19:43:57.410 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:43:57.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:43:57.413 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T19:43:57.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T19:43:57.411+0000 7f20da9b5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 62492) UID: 1000 2021-02-20T19:43:57.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:43:59.413 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:43:59.414 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:43:59.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:43:59.415 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:43:59.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:43:59.415 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:43:59.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:43:59.418 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T19:44:27.605 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2854408398494013) > weight (0.25) 2021-02-20T19:44:27.616 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4168329349104174) > weight (0.25) 2021-02-20T19:44:27.616 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9699620809542286) > weight (0.25) 2021-02-20T19:44:27.616 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7269495309454393) > weight (0.25) 2021-02-20T19:44:27.616 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T19:44:51.917 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T19:44:51.885+0000 7fb8259e4700 -1 reset not still connected to 0x5578525098e0 2021-02-20T19:44:51.918 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T19:44:51.885+0000 7fb8259e4700 -1 reset not still connected to 0x55785f2abd40 2021-02-20T19:45:06.419 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7570770364781435) > weight (0.25) 2021-02-20T19:45:06.431 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:45:06.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:45:06.433 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T19:45:06.433 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:45:06.433 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:45:06.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T19:45:06.425+0000 7f34c64c5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 62118) UID: 1000 2021-02-20T19:45:06.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:45:12.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:45:12.536 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:45:12.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:45:12.536 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:45:12.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:45:12.540 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T19:46:09.617 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:46:09.625 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:46:09.626 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T19:46:09.627 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T19:46:09.623+0000 7f35def8e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 62584) UID: 1000 2021-02-20T19:46:09.827 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:46:10.627 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:46:10.628 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:46:10.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:46:10.628 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:46:10.629 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:46:10.629 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:46:10.632 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:46:10.632 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T19:46:35.541 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8115094920061173) > weight (0.25) 2021-02-20T19:46:35.550 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7151242428893276) > weight (0.25) 2021-02-20T19:46:35.550 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.33287148878318906) > weight (0.25) 2021-02-20T19:46:35.550 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.771674171479495) > weight (0.25) 2021-02-20T19:46:35.551 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T19:47:18.633 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9155785224427556) > weight (0.25) 2021-02-20T19:47:18.645 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8719940040544909) > weight (0.25) 2021-02-20T19:47:18.645 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.25013986407980515) > weight (0.25) 2021-02-20T19:47:18.645 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9025285821666987) > weight (0.25) 2021-02-20T19:47:18.646 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T19:48:34.552 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8360072586767757) > weight (0.25) 2021-02-20T19:48:34.617 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8095085119950336) > weight (0.25) 2021-02-20T19:48:34.618 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6697394271326453) > weight (0.25) 2021-02-20T19:48:34.618 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5097852019818045) > weight (0.25) 2021-02-20T19:48:34.618 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T19:48:56.647 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5745928347904478) > weight (0.25) 2021-02-20T19:48:56.648 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.928093844207425) > weight (0.25) 2021-02-20T19:48:56.648 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6166073959319556) > weight (0.25) 2021-02-20T19:48:56.648 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9693483732206337) > weight (0.25) 2021-02-20T19:48:56.648 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T19:49:45.620 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5131177152615957) > weight (0.25) 2021-02-20T19:49:45.627 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6060507052247253) > weight (0.25) 2021-02-20T19:49:45.628 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4412554778896983) > weight (0.25) 2021-02-20T19:49:45.628 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8115117781230451) > weight (0.25) 2021-02-20T19:49:45.628 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T19:50:13.649 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.36602000801295076) > weight (0.25) 2021-02-20T19:50:13.650 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.39083909369822956) > weight (0.25) 2021-02-20T19:50:13.650 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4139237934625587) > weight (0.25) 2021-02-20T19:50:13.650 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9465961474656224) > weight (0.25) 2021-02-20T19:50:13.650 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T19:51:01.629 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.42150003811753334) > weight (0.25) 2021-02-20T19:51:01.630 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.30640567964524557) > weight (0.25) 2021-02-20T19:51:01.630 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7535895942019084) > weight (0.25) 2021-02-20T19:51:01.631 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5935444170802454) > weight (0.25) 2021-02-20T19:51:01.631 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T19:51:25.651 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:51:25.669 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:51:25.669 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T19:51:25.670 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T19:51:25.667+0000 7f5c2b2f8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 62956) UID: 1000 2021-02-20T19:51:25.871 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:51:27.670 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:51:27.670 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:51:27.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:51:27.671 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:51:27.671 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:51:27.672 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:51:27.676 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:51:27.676 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T19:51:40.693 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T19:51:40.690+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b66a6a90 2021-02-20T19:51:40.855 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T19:51:40.690+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b9ff68a0 2021-02-20T19:52:33.632 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6021456602066404) > weight (0.25) 2021-02-20T19:52:33.639 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.41985498065399496) > weight (0.25) 2021-02-20T19:52:33.640 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4518507589774766) > weight (0.25) 2021-02-20T19:52:33.640 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.77169260840156) > weight (0.25) 2021-02-20T19:52:33.640 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T19:53:09.678 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:53:09.687 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:53:09.688 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T19:53:09.689 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T19:53:09.688+0000 7f8282869700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 63051) UID: 1000 2021-02-20T19:53:09.888 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:53:14.689 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:53:14.690 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:53:14.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:53:14.691 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:53:14.691 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:53:14.691 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:53:14.694 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:53:14.694 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T19:54:11.641 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:54:11.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:54:11.650 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T19:54:11.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T19:54:11.646+0000 7fc2bda46700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 62770) UID: 1000 2021-02-20T19:54:11.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:54:20.651 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:54:20.651 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:54:20.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:54:20.652 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:54:20.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:54:20.653 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:54:20.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:54:20.655 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T19:54:47.695 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:54:47.991 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:54:47.991 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T19:54:47.992 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T19:54:47.985+0000 7fc8270e1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 63143) UID: 1000 2021-02-20T19:54:48.192 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:54:54.992 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:54:55.000 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:54:55.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:54:55.001 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:54:55.001 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:54:55.002 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:54:55.005 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:54:55.005 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T19:55:24.656 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:55:24.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:55:24.658 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T19:55:24.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T19:55:24.656+0000 7f7c13462700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 63234) UID: 1000 2021-02-20T19:55:24.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:55:32.659 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:55:32.659 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:55:32.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:55:32.660 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:55:32.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:55:32.661 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:55:32.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:55:32.666 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T19:55:58.007 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6317519245619485) > weight (0.25) 2021-02-20T19:55:58.015 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5834877278539834) > weight (0.25) 2021-02-20T19:55:58.015 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3687662554307243) > weight (0.25) 2021-02-20T19:55:58.016 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3649422270121002) > weight (0.25) 2021-02-20T19:55:58.016 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T19:57:04.647 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T19:57:04.643+0000 7fb8259e4700 -1 reset not still connected to 0x557863b46e50 2021-02-20T19:57:04.766 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T19:57:04.647+0000 7fb8259e4700 -1 reset not still connected to 0x557865e34f70 2021-02-20T19:57:13.668 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8797024206085508) > weight (0.25) 2021-02-20T19:57:13.675 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7281816333234907) > weight (0.25) 2021-02-20T19:57:13.676 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8678076278516067) > weight (0.25) 2021-02-20T19:57:13.676 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8621308506034833) > weight (0.25) 2021-02-20T19:57:13.676 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T19:57:27.017 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:57:27.030 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:57:27.032 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T19:57:27.033 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T19:57:27.029+0000 7fcecf151700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 63325) UID: 1000 2021-02-20T19:57:27.232 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:57:28.033 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:57:28.034 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:57:28.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:57:28.035 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:57:28.035 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:57:28.036 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:57:28.039 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:57:28.039 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T19:58:49.040 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4326205510391412) > weight (0.25) 2021-02-20T19:58:49.055 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.36327285055079395) > weight (0.25) 2021-02-20T19:58:49.056 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T19:58:49.056 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T19:58:49.057 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T19:58:49.058 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T19:58:49.055+0000 7f98ca9d2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 62209) UID: 1000 2021-02-20T19:58:49.258 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:58:52.058 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T19:58:52.062 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:58:52.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:58:52.063 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T19:58:52.063 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T19:58:52.064 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T19:58:52.067 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T19:58:52.067 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T19:58:55.677 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5329962075783541) > weight (0.25) 2021-02-20T19:58:55.678 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.762614402578771) > weight (0.25) 2021-02-20T19:58:55.678 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5131263577745492) > weight (0.25) 2021-02-20T19:58:55.678 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7177980130226137) > weight (0.25) 2021-02-20T19:58:55.679 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T20:00:17.068 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.820822717094848) > weight (0.25) 2021-02-20T20:00:17.822 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8203102002272203) > weight (0.25) 2021-02-20T20:00:17.823 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4471503314398171) > weight (0.25) 2021-02-20T20:00:17.823 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8004260138888507) > weight (0.25) 2021-02-20T20:00:17.823 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T20:00:27.679 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T20:00:27.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T20:00:27.689 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T20:00:27.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T20:00:27.688+0000 7f3fad2c9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 63420) UID: 1000 2021-02-20T20:00:27.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:00:28.690 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T20:00:28.690 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:00:28.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:00:28.691 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T20:00:28.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T20:00:28.692 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T20:00:28.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T20:00:28.695 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T20:01:18.825 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7026100831003466) > weight (0.25) 2021-02-20T20:01:18.837 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.34336204791847436) > weight (0.25) 2021-02-20T20:01:18.837 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3220981939071622) > weight (0.25) 2021-02-20T20:01:18.838 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T20:01:18.839 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T20:01:18.839 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T20:01:18.840 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T20:01:18.840 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:01:18.844 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T20:01:18.839+0000 7f8590dc5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 62401) UID: 1000 2021-02-20T20:01:19.044 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:01:24.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:01:24.952 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T20:01:24.952 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T20:01:24.955 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T20:01:24.959 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T20:01:24.959 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T20:02:17.697 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.47397567347471836) > weight (0.25) 2021-02-20T20:02:17.708 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3700821019117472) > weight (0.25) 2021-02-20T20:02:17.708 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3206765747456325) > weight (0.25) 2021-02-20T20:02:17.708 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T20:02:17.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T20:02:17.709 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T20:02:17.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T20:02:17.709+0000 7f9944f2c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 60914) UID: 1000 2021-02-20T20:02:18.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:02:21.712 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T20:02:21.713 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:02:21.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:02:21.714 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T20:02:21.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T20:02:21.715 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T20:02:21.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T20:02:21.718 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T20:02:22.738 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T20:02:22.729+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a4750820 2021-02-20T20:02:22.741 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T20:02:22.733+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a9c709c0 2021-02-20T20:02:22.741 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T20:02:22.733+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a9c73810 2021-02-20T20:02:22.742 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T20:02:22.733+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6abc092b0 2021-02-20T20:02:22.742 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T20:02:22.733+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6abc0ad80 2021-02-20T20:02:22.743 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T20:02:22.733+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b66a8cb0 2021-02-20T20:02:22.743 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T20:02:22.733+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b9ff5a00 2021-02-20T20:03:19.960 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5165071315745859) > weight (0.25) 2021-02-20T20:03:19.970 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9857457077769676) > weight (0.25) 2021-02-20T20:03:19.970 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3900371900820878) > weight (0.25) 2021-02-20T20:03:19.971 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3727172117903935) > weight (0.25) 2021-02-20T20:03:19.971 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T20:03:55.718 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7855620958981169) > weight (0.25) 2021-02-20T20:03:55.719 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5729554220881223) > weight (0.25) 2021-02-20T20:03:55.719 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.38559317899225964) > weight (0.25) 2021-02-20T20:03:55.719 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T20:03:55.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T20:03:55.720 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T20:03:55.721 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T20:03:55.721 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:03:55.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T20:03:55.718+0000 7f20d2b00700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 63947) UID: 1000 2021-02-20T20:03:55.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:04:01.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:04:06.345 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T20:04:06.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T20:04:06.346 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T20:04:06.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T20:04:06.350 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T20:05:16.972 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T20:05:16.984 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T20:05:16.985 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T20:05:16.986 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T20:05:16.984+0000 7f141d614700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 63511) UID: 1000 2021-02-20T20:05:17.186 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:05:23.985 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T20:05:23.986 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:05:23.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:05:23.987 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T20:05:23.987 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T20:05:23.988 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T20:05:23.990 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T20:05:23.991 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T20:05:45.351 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T20:05:45.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T20:05:45.360 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T20:05:45.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T20:05:45.359+0000 7f128c380700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 63764) UID: 1000 2021-02-20T20:05:45.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:05:49.362 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T20:05:49.367 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:05:49.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:05:49.368 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T20:05:49.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T20:05:49.369 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T20:05:49.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T20:05:49.372 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T20:06:46.992 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T20:06:46.993 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T20:06:46.993 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T20:06:46.994 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T20:06:46.990+0000 7f0735f13700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 64141) UID: 1000 2021-02-20T20:06:47.195 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:06:54.995 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T20:06:55.002 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:06:55.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:06:55.003 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T20:06:55.003 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T20:06:55.004 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T20:06:55.007 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T20:06:55.007 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T20:07:41.373 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T20:07:41.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T20:07:41.381 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T20:07:41.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T20:07:41.380+0000 7f601bb10700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 64232) UID: 1000 2021-02-20T20:07:41.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:07:42.382 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T20:07:42.383 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:07:42.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:07:42.383 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T20:07:42.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T20:07:42.384 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T20:07:42.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T20:07:42.387 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T20:08:49.009 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.33371422942304685) > weight (0.25) 2021-02-20T20:08:49.018 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.527669416165566) > weight (0.25) 2021-02-20T20:08:49.018 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.577472535269935) > weight (0.25) 2021-02-20T20:08:49.019 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4467757240468231) > weight (0.25) 2021-02-20T20:08:49.019 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T20:09:37.388 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6865359568983841) > weight (0.25) 2021-02-20T20:09:37.396 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.898436777668215) > weight (0.25) 2021-02-20T20:09:37.396 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T20:09:37.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T20:09:37.398 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T20:09:37.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T20:09:37.393+0000 7f355351c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 61192) UID: 1000 2021-02-20T20:09:37.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:09:45.399 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T20:09:45.407 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:09:45.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:09:45.408 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T20:09:45.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T20:09:45.409 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T20:09:45.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T20:09:45.412 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T20:09:55.019 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T20:09:55.020 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T20:09:55.020 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T20:09:55.021 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T20:09:55.021+0000 7f82f7431700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 64332) UID: 1000 2021-02-20T20:09:55.222 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:10:04.021 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T20:10:04.021 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:10:04.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:10:04.022 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T20:10:04.022 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T20:10:04.023 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T20:10:04.025 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T20:10:04.026 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T20:11:16.413 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T20:11:16.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T20:11:16.424 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T20:11:16.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T20:11:16.423+0000 7ffa0231e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 64423) UID: 1000 2021-02-20T20:11:16.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:11:24.027 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3338410553181793) > weight (0.25) 2021-02-20T20:11:24.028 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8136842067486292) > weight (0.25) 2021-02-20T20:11:24.028 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T20:11:24.029 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T20:11:24.029 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T20:11:24.030 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T20:11:24.026+0000 7f72e6c37700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 63604) UID: 1000 2021-02-20T20:11:24.230 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:11:24.425 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T20:11:24.425 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:11:24.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:11:24.426 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T20:11:24.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T20:11:24.427 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T20:11:24.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T20:11:24.430 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T20:11:29.030 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T20:11:29.030 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:11:29.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:11:29.031 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T20:11:29.032 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T20:11:29.032 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T20:11:29.034 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T20:11:29.035 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T20:12:50.036 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3582346418444674) > weight (0.25) 2021-02-20T20:12:50.043 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T20:12:50.044 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T20:12:50.045 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T20:12:50.046 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T20:12:50.044+0000 7febe3df6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 62676) UID: 1000 2021-02-20T20:12:50.446 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:12:56.046 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T20:12:56.046 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:12:56.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:12:56.047 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T20:12:56.047 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T20:12:56.048 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T20:12:56.051 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T20:12:56.051 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T20:13:06.431 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3500529901649704) > weight (0.25) 2021-02-20T20:13:06.438 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8834063874986982) > weight (0.25) 2021-02-20T20:13:06.438 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T20:13:06.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T20:13:06.440 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T20:13:06.440 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T20:13:06.440 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:13:06.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T20:13:06.436+0000 7fdfbc1cc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 64515) UID: 1000 2021-02-20T20:13:06.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:13:12.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:13:12.543 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T20:13:12.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T20:13:12.543 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T20:13:12.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T20:13:12.546 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T20:14:04.052 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T20:14:04.062 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T20:14:04.063 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T20:14:04.064 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T20:14:04.058+0000 7f1e971f5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 64607) UID: 1000 2021-02-20T20:14:04.263 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:14:08.064 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T20:14:08.071 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:14:08.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:14:08.071 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T20:14:08.072 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T20:14:08.072 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T20:14:08.075 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T20:14:08.075 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T20:15:11.548 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5153475057628233) > weight (0.25) 2021-02-20T20:15:11.556 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.25344547926619654) > weight (0.25) 2021-02-20T20:15:11.557 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.40709430359301413) > weight (0.25) 2021-02-20T20:15:11.557 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5042799027507819) > weight (0.25) 2021-02-20T20:15:11.557 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T20:15:43.076 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3800446563874441) > weight (0.25) 2021-02-20T20:15:43.084 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T20:15:43.085 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T20:15:43.086 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T20:15:43.086 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T20:15:43.086 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:15:43.087 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T20:15:43.083+0000 7f0f704b5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 64882) UID: 1000 2021-02-20T20:15:43.287 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:15:49.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:15:49.189 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T20:15:49.190 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T20:15:49.190 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T20:15:49.194 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T20:15:49.194 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T20:16:43.558 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T20:16:43.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T20:16:43.564 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T20:16:43.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T20:16:43.562+0000 7f4185685700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 64698) UID: 1000 2021-02-20T20:16:43.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:16:52.565 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T20:16:52.572 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:16:52.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:16:52.573 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T20:16:52.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T20:16:52.574 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T20:16:52.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T20:16:52.577 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T20:17:19.195 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4031800824747722) > weight (0.25) 2021-02-20T20:17:19.196 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.28912384980067063) > weight (0.25) 2021-02-20T20:17:19.196 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.26886297851544483) > weight (0.25) 2021-02-20T20:17:19.196 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T20:17:19.197 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T20:17:19.197 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T20:17:19.198 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T20:17:19.193+0000 7f9b0a69b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 63856) UID: 1000 2021-02-20T20:17:19.398 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:17:28.198 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T20:17:28.198 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:17:28.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:17:28.199 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T20:17:28.199 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T20:17:28.200 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T20:17:28.202 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T20:17:28.203 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T20:18:27.578 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5166022588370669) > weight (0.25) 2021-02-20T20:18:27.588 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3930218999387103) > weight (0.25) 2021-02-20T20:18:27.588 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3046357115943562) > weight (0.25) 2021-02-20T20:18:27.588 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4542643881887204) > weight (0.25) 2021-02-20T20:18:27.588 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T20:19:13.204 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5342341007947541) > weight (0.25) 2021-02-20T20:19:13.214 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7275604192911048) > weight (0.25) 2021-02-20T20:19:13.214 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9294632954207395) > weight (0.25) 2021-02-20T20:19:13.214 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.38064377019328466) > weight (0.25) 2021-02-20T20:19:13.215 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T20:19:40.590 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3550206707772031) > weight (0.25) 2021-02-20T20:19:40.591 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.53821981680768) > weight (0.25) 2021-02-20T20:19:40.591 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3942638700095369) > weight (0.25) 2021-02-20T20:19:40.591 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T20:19:40.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T20:19:40.592 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T20:19:40.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T20:19:40.589+0000 7f1b4e83d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 64047) UID: 1000 2021-02-20T20:19:40.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:19:43.593 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T20:19:43.594 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:19:43.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:19:43.595 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T20:19:43.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T20:19:43.596 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T20:19:43.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T20:19:43.599 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T20:19:45.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T20:19:45.461+0000 7f6ad3aa2700 -1 rbd::mirror::ImageReplayer: 0x55567b726a00 [3/eafa574f-c2c8-4659-b559-be75ec7d56bb] register_admin_socket_hook: error registering admin socket commands 2021-02-20T20:19:45.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T20:19:45.465+0000 7f6ad32a1700 -1 rbd::mirror::ImageReplayer: 0x55567b726a00 [3/eafa574f-c2c8-4659-b559-be75ec7d56bb] register_admin_socket_hook: error registering admin socket commands 2021-02-20T20:19:45.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T20:19:45.465+0000 7f6ad3aa2700 -1 rbd::mirror::ImageReplayer: 0x555677731200 [3/4cba354e-99d2-4859-a4d0-baeec2203656] register_admin_socket_hook: error registering admin socket commands 2021-02-20T20:19:45.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T20:19:45.465+0000 7f6ad32a1700 -1 rbd::mirror::ImageReplayer: 0x555677731200 [3/4cba354e-99d2-4859-a4d0-baeec2203656] register_admin_socket_hook: error registering admin socket commands 2021-02-20T20:20:33.216 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6696004746544311) > weight (0.25) 2021-02-20T20:20:33.226 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T20:20:33.226 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T20:20:33.227 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T20:20:33.228 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T20:20:33.224+0000 7f81ee116700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 65159) UID: 1000 2021-02-20T20:20:33.428 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:20:37.227 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T20:20:37.228 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:20:37.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:20:37.229 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T20:20:37.229 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T20:20:37.229 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T20:20:37.232 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T20:20:37.232 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T20:21:12.600 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.977443258248779) > weight (0.25) 2021-02-20T20:21:12.601 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7351721602662292) > weight (0.25) 2021-02-20T20:21:12.601 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T20:21:12.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T20:21:12.602 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T20:21:12.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T20:21:12.599+0000 7fc4517d4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 64973) UID: 1000 2021-02-20T20:21:12.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:21:17.603 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T20:21:17.604 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:21:17.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:21:17.604 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T20:21:17.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T20:21:17.605 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T20:21:17.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T20:21:17.608 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T20:22:23.233 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9420274564795316) > weight (0.25) 2021-02-20T20:22:23.240 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.688905075113325) > weight (0.25) 2021-02-20T20:22:23.240 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6832311278750773) > weight (0.25) 2021-02-20T20:22:23.241 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.30689680408368936) > weight (0.25) 2021-02-20T20:22:23.241 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T20:23:14.609 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9931946525730827) > weight (0.25) 2021-02-20T20:23:14.610 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7006447683907917) > weight (0.25) 2021-02-20T20:23:14.610 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.391698127328139) > weight (0.25) 2021-02-20T20:23:14.610 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T20:23:14.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T20:23:14.612 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T20:23:14.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T20:23:14.607+0000 7ff302cc3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 65437) UID: 1000 2021-02-20T20:23:14.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:23:20.613 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T20:23:20.614 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:23:20.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:23:20.615 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T20:23:20.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T20:23:20.615 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T20:23:20.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T20:23:20.619 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T20:24:22.243 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T20:24:22.250 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T20:24:22.250 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T20:24:22.251 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T20:24:22.246+0000 7f3224c92700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 65065) UID: 1000 2021-02-20T20:24:22.452 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:24:26.470 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T20:24:26.465+0000 7f13c28a4700 -1 rbd::mirror::ImageReplayer: 0x5575a9b50c80 [3/172baefb-5ce5-45e0-997e-249560de6fbd] start: already running 2021-02-20T20:24:27.251 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T20:24:27.252 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:24:27.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:24:27.253 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T20:24:27.253 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T20:24:27.253 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T20:24:27.256 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T20:24:27.256 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T20:24:27.591 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T20:24:27.585+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a3b83040 2021-02-20T20:24:27.591 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T20:24:27.585+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a4752b10 2021-02-20T20:24:27.592 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T20:24:27.585+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a47539b0 2021-02-20T20:24:55.620 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8099705244786608) > weight (0.25) 2021-02-20T20:24:55.625 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T20:24:55.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T20:24:55.626 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T20:24:55.626 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T20:24:55.626 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:24:55.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T20:24:55.625+0000 7fbeca705700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 62865) UID: 1000 2021-02-20T20:24:55.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:25:01.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:25:01.729 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T20:25:01.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T20:25:01.729 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T20:25:01.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T20:25:01.733 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T20:25:53.258 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6407064930410283) > weight (0.25) 2021-02-20T20:25:53.281 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6647018708510984) > weight (0.25) 2021-02-20T20:25:53.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7008129615272802) > weight (0.25) 2021-02-20T20:25:53.282 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T20:25:53.283 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T20:25:53.284 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T20:25:53.286 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T20:25:53.283+0000 7fe8fc51f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 65345) UID: 1000 2021-02-20T20:25:53.486 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:26:00.286 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T20:26:00.286 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:26:00.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:26:00.287 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T20:26:00.287 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T20:26:00.288 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T20:26:00.291 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T20:26:00.291 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T20:26:58.733 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8962863376992555) > weight (0.25) 2021-02-20T20:26:58.738 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T20:26:58.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T20:26:58.739 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T20:26:58.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T20:26:58.733+0000 7fd33a6b6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 65906) UID: 1000 2021-02-20T20:26:58.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:27:05.740 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T20:27:05.740 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:27:05.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:27:05.741 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T20:27:05.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T20:27:05.742 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T20:27:05.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T20:27:05.745 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T20:27:17.292 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T20:27:17.293 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T20:27:17.293 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T20:27:17.294 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T20:27:17.293+0000 7f0aad08c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 65813) UID: 1000 2021-02-20T20:27:17.495 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:27:21.294 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T20:27:21.294 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:27:21.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:27:21.295 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T20:27:21.295 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T20:27:21.295 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T20:27:21.298 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T20:27:21.298 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T20:28:06.746 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.905498242450491) > weight (0.25) 2021-02-20T20:28:06.747 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T20:28:06.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T20:28:06.748 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T20:28:06.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T20:28:06.748+0000 7fa2814d3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 66089) UID: 1000 2021-02-20T20:28:06.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:28:14.749 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T20:28:14.750 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:28:14.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:28:14.751 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T20:28:14.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T20:28:14.751 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T20:28:14.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T20:28:14.754 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T20:28:52.299 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7857260990674989) > weight (0.25) 2021-02-20T20:28:52.306 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5013700875781788) > weight (0.25) 2021-02-20T20:28:52.306 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9290819337096233) > weight (0.25) 2021-02-20T20:28:52.307 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9680317995408286) > weight (0.25) 2021-02-20T20:28:52.307 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T20:30:12.756 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T20:30:12.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T20:30:12.767 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T20:30:12.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T20:30:12.764+0000 7f6ad42a3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 65250) UID: 1000 2021-02-20T20:30:13.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:30:15.768 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T20:30:15.769 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:30:15.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:30:15.770 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T20:30:15.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T20:30:15.771 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T20:30:15.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T20:30:15.774 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T20:30:17.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T20:30:17.120+0000 7f505f40f700 -1 rbd::mirror::ImageReplayer: 0x56540d800f00 [3/172baefb-5ce5-45e0-997e-249560de6fbd] start: already running 2021-02-20T20:30:17.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T20:30:17.120+0000 7f505fc10700 -1 rbd::mirror::ImageReplayer: 0x56540d801180 [3/1a170226-518b-4a75-85a8-6fbdc9b5d9f0] start: already running 2021-02-20T20:30:17.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T20:30:17.120+0000 7f505fc10700 -1 rbd::mirror::ImageReplayer: 0x56540d801400 [3/205dce0d-bc7c-41b6-8ed9-87e6a63d1a70] start: already running 2021-02-20T20:30:17.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T20:30:17.120+0000 7f505fc10700 -1 rbd::mirror::ImageReplayer: 0x56540d801680 [3/2464eeeb-6954-426d-a8e5-129943cea55c] start: already running 2021-02-20T20:30:17.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T20:30:17.120+0000 7f505fc10700 -1 rbd::mirror::ImageReplayer: 0x56540d801900 [3/3ab5666f-763c-442d-bc17-ea037dcc3a42] start: already running 2021-02-20T20:30:17.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T20:30:17.120+0000 7f505fc10700 -1 rbd::mirror::ImageReplayer: 0x56540d801b80 [3/3c6940ee-a642-422e-aafb-e3920bba87cf] start: already running 2021-02-20T20:30:17.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T20:30:17.120+0000 7f505fc10700 -1 rbd::mirror::ImageReplayer: 0x56540d801e00 [3/3d38caf8-6420-4f62-95dc-628caa5144da] start: already running 2021-02-20T20:30:17.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T20:30:17.120+0000 7f505f40f700 -1 rbd::mirror::ImageReplayer: 0x56540d802300 [3/4c862af6-d1da-4215-b6b8-461f3883b104] start: already running 2021-02-20T20:30:17.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T20:30:17.120+0000 7f505f40f700 -1 rbd::mirror::ImageReplayer: 0x56540d802580 [3/4cba354e-99d2-4859-a4d0-baeec2203656] start: already running 2021-02-20T20:30:17.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T20:30:17.120+0000 7f505f40f700 -1 rbd::mirror::ImageReplayer: 0x56540d802800 [3/567ace7d-44e4-4d42-b47a-38bd35ec0e11] start: already running 2021-02-20T20:30:17.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T20:30:17.120+0000 7f505fc10700 -1 rbd::mirror::ImageReplayer: 0x56540d802a80 [3/5786ec48-0da5-470d-b6a6-ddd71a666657] start: already running 2021-02-20T20:30:17.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T20:30:17.120+0000 7f505fc10700 -1 rbd::mirror::ImageReplayer: 0x56540d802d00 [3/5c036645-b242-4fa8-bde0-d5b28dda3fd3] start: already running 2021-02-20T20:30:18.308 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.580454397851079) > weight (0.25) 2021-02-20T20:30:18.308 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5419672059237517) > weight (0.25) 2021-02-20T20:30:18.309 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T20:30:18.309 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T20:30:18.310 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T20:30:18.310 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T20:30:18.308+0000 7f13c30a5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 64789) UID: 1000 2021-02-20T20:30:18.711 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:30:22.558 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T20:30:22.556+0000 7f4a1bb72700 -1 reset not still connected to 0x55fc74aa3790 2021-02-20T20:30:22.562 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T20:30:22.560+0000 7f4a1bb72700 -1 reset not still connected to 0x55fc98a90080 2021-02-20T20:30:25.311 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T20:30:25.311 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:30:25.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:30:25.312 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T20:30:25.312 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T20:30:25.312 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T20:30:25.315 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T20:30:25.315 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T20:31:07.516 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T20:31:07.511+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ca081b50 2021-02-20T20:31:07.521 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T20:31:07.515+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ca081dc0 2021-02-20T20:31:07.521 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T20:31:07.515+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6e58aa270 2021-02-20T20:31:50.774 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T20:31:50.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T20:31:50.784 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T20:31:50.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T20:31:50.782+0000 7ff26ea04700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 66367) UID: 1000 2021-02-20T20:31:50.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:31:51.317 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T20:31:51.317 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T20:31:51.318 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T20:31:51.318 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T20:31:51.314+0000 7fa441548700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 66180) UID: 1000 2021-02-20T20:31:51.519 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:31:53.319 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T20:31:53.320 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:31:53.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:31:53.320 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T20:31:53.321 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T20:31:53.321 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T20:31:53.324 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T20:31:53.324 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T20:31:57.786 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T20:31:57.786 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:31:57.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:31:57.787 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T20:31:57.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T20:31:57.788 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T20:31:57.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T20:31:57.791 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T20:33:26.792 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9040367608401421) > weight (0.25) 2021-02-20T20:33:26.793 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T20:33:26.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T20:33:26.794 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T20:33:26.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T20:33:26.791+0000 7fd22b42b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 66273) UID: 1000 2021-02-20T20:33:26.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:33:30.795 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T20:33:30.796 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:33:30.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:33:30.797 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T20:33:30.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T20:33:30.797 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T20:33:30.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T20:33:30.800 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T20:33:38.326 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9751369417032811) > weight (0.25) 2021-02-20T20:33:38.326 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T20:33:38.327 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T20:33:38.327 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T20:33:38.327 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T20:33:38.327 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:33:38.328 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T20:33:38.323+0000 7ff8b8fe2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 65537) UID: 1000 2021-02-20T20:33:38.528 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:33:44.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:33:44.430 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T20:33:44.430 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T20:33:44.431 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T20:33:44.433 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T20:33:44.434 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T20:34:33.802 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4975466823089335) > weight (0.25) 2021-02-20T20:34:33.802 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T20:34:33.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T20:34:33.804 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T20:34:33.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T20:34:33.797+0000 7f4ecdd15700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 66739) UID: 1000 2021-02-20T20:34:34.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:34:37.805 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T20:34:37.805 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:34:37.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:34:37.806 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T20:34:37.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T20:34:37.807 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T20:34:37.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T20:34:37.810 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T20:35:04.435 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5649724508273091) > weight (0.25) 2021-02-20T20:35:04.436 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T20:35:04.437 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T20:35:04.438 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T20:35:04.439 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T20:35:04.436+0000 7faf205b9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 66830) UID: 1000 2021-02-20T20:35:04.639 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:35:12.439 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T20:35:12.439 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:35:12.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:35:12.440 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T20:35:12.440 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T20:35:12.441 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T20:35:12.444 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T20:35:12.444 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T20:36:00.811 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.38983770658294514) > weight (0.25) 2021-02-20T20:36:00.812 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T20:36:00.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T20:36:00.813 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T20:36:00.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T20:36:00.811+0000 7f1cd8c51700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 66923) UID: 1000 2021-02-20T20:36:01.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:36:01.814 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T20:36:01.815 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:36:01.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:36:01.815 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T20:36:01.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T20:36:01.816 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T20:36:01.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T20:36:01.819 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T20:36:48.445 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6312475895247225) > weight (0.25) 2021-02-20T20:36:48.452 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T20:36:48.453 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T20:36:48.453 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T20:36:48.454 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T20:36:48.450+0000 7fc6b7325700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 67018) UID: 1000 2021-02-20T20:36:48.654 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:36:52.453 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T20:36:52.454 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:36:52.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:36:52.455 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T20:36:52.455 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T20:36:52.456 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T20:36:52.460 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T20:36:52.460 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T20:37:51.820 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T20:37:51.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T20:37:51.830 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T20:37:51.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T20:37:51.828+0000 7f5e9936d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 66648) UID: 1000 2021-02-20T20:37:52.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:38:00.830 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T20:38:00.831 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:38:00.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:38:00.832 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T20:38:00.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T20:38:00.832 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T20:38:00.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T20:38:00.835 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T20:38:03.461 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.416612766289998) > weight (0.25) 2021-02-20T20:38:03.461 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.36702083181511114) > weight (0.25) 2021-02-20T20:38:03.462 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6603053267299298) > weight (0.25) 2021-02-20T20:38:03.462 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6902831630196656) > weight (0.25) 2021-02-20T20:38:03.462 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T20:39:27.837 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7419898095520926) > weight (0.25) 2021-02-20T20:39:27.838 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.45406317242394) > weight (0.25) 2021-02-20T20:39:27.838 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.41976403945791807) > weight (0.25) 2021-02-20T20:39:27.838 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T20:39:27.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T20:39:27.840 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T20:39:27.840 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T20:39:27.840 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:39:27.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T20:39:27.833+0000 7ff9ad1d9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 65721) UID: 1000 2021-02-20T20:39:28.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:39:33.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:39:33.943 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T20:39:33.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T20:39:33.944 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T20:39:33.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T20:39:33.948 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T20:39:39.463 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.48058352465934084) > weight (0.25) 2021-02-20T20:39:39.464 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5519094127848596) > weight (0.25) 2021-02-20T20:39:39.464 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9296838049613801) > weight (0.25) 2021-02-20T20:39:39.464 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6202626046615316) > weight (0.25) 2021-02-20T20:39:39.465 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T20:41:02.466 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9066497567589323) > weight (0.25) 2021-02-20T20:41:02.476 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T20:41:02.477 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T20:41:02.478 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T20:41:02.478 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T20:41:02.475+0000 7febf17bf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 67200) UID: 1000 2021-02-20T20:41:02.679 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:41:03.479 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T20:41:03.479 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:41:03.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:41:03.480 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T20:41:03.481 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T20:41:03.481 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T20:41:03.484 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T20:41:03.484 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T20:41:07.950 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6474220529864764) > weight (0.25) 2021-02-20T20:41:07.951 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8172393489845182) > weight (0.25) 2021-02-20T20:41:07.951 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2905256807747263) > weight (0.25) 2021-02-20T20:41:07.951 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6116055875188805) > weight (0.25) 2021-02-20T20:41:07.952 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T20:42:13.952 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8370826457354648) > weight (0.25) 2021-02-20T20:42:13.962 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.27920039662222307) > weight (0.25) 2021-02-20T20:42:13.962 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9892136836919379) > weight (0.25) 2021-02-20T20:42:13.962 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7920528932037127) > weight (0.25) 2021-02-20T20:42:13.962 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T20:42:15.486 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8131525534031185) > weight (0.25) 2021-02-20T20:42:15.486 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T20:42:15.487 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T20:42:15.487 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T20:42:15.488 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T20:42:15.481+0000 7f34610f3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 67476) UID: 1000 2021-02-20T20:42:15.689 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:42:24.488 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T20:42:24.493 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:42:24.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:42:24.494 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T20:42:24.494 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T20:42:24.495 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T20:42:24.499 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T20:42:24.499 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T20:43:32.501 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.777498981706157) > weight (0.25) 2021-02-20T20:43:32.510 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9866173133316908) > weight (0.25) 2021-02-20T20:43:32.510 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.626789851393488) > weight (0.25) 2021-02-20T20:43:32.511 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7043767994646916) > weight (0.25) 2021-02-20T20:43:32.511 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T20:44:02.963 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3493527477266347) > weight (0.25) 2021-02-20T20:44:10.537 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.550380268271131) > weight (0.25) 2021-02-20T20:44:10.553 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5982298327836774) > weight (0.25) 2021-02-20T20:44:10.554 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T20:44:10.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T20:44:10.555 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T20:44:10.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T20:44:10.554+0000 7f85bf407700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 67384) UID: 1000 2021-02-20T20:44:10.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:44:16.556 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T20:44:16.557 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:44:16.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:44:16.558 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T20:44:16.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T20:44:16.558 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T20:44:16.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T20:44:16.562 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T20:44:44.512 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9338518956960786) > weight (0.25) 2021-02-20T20:44:44.520 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3084689058835317) > weight (0.25) 2021-02-20T20:44:44.520 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8468111376725743) > weight (0.25) 2021-02-20T20:44:44.520 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9754247439980787) > weight (0.25) 2021-02-20T20:44:44.521 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T20:45:58.521 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6629645464536652) > weight (0.25) 2021-02-20T20:45:58.528 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5610648438197476) > weight (0.25) 2021-02-20T20:45:58.528 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T20:45:58.529 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T20:45:58.530 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T20:45:58.531 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T20:45:58.527+0000 7fe3a86d7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 66466) UID: 1000 2021-02-20T20:45:58.731 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:46:05.531 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T20:46:05.535 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:46:05.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:46:05.536 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T20:46:05.536 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T20:46:05.536 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T20:46:05.539 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T20:46:05.539 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T20:46:07.562 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4852789902173811) > weight (0.25) 2021-02-20T20:46:07.563 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3948835009771049) > weight (0.25) 2021-02-20T20:46:07.563 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5034011522391142) > weight (0.25) 2021-02-20T20:46:07.563 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T20:46:07.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T20:46:07.564 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T20:46:07.565 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T20:46:07.565 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:46:07.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T20:46:07.563+0000 7fe07d66b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 67659) UID: 1000 2021-02-20T20:46:07.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:46:13.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:46:13.666 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T20:46:13.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T20:46:13.667 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T20:46:13.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T20:46:13.670 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T20:47:08.976 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T20:47:08.969+0000 7f03ccd38700 -1 reset not still connected to 0x556b8287f520 2021-02-20T20:47:08.977 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T20:47:08.969+0000 7f03ccd38700 -1 reset not still connected to 0x556b828815a0 2021-02-20T20:47:08.979 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T20:47:08.973+0000 7f03ccd38700 -1 reset not still connected to 0x556b82881810 2021-02-20T20:47:08.986 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T20:47:08.977+0000 7f03ccd38700 -1 reset not still connected to 0x556b899095a0 2021-02-20T20:47:08.986 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T20:47:08.977+0000 7f03ccd38700 -1 reset not still connected to 0x556b8baa9930 2021-02-20T20:47:08.987 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T20:47:08.977+0000 7f03ccd38700 -1 reset not still connected to 0x556b8baa9e10 2021-02-20T20:47:08.987 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T20:47:08.977+0000 7f03ccd38700 -1 reset not still connected to 0x556b8baaa7d0 2021-02-20T20:47:08.988 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T20:47:08.977+0000 7f03ccd38700 -1 reset not still connected to 0x556b8baaad80 2021-02-20T20:47:08.988 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T20:47:08.977+0000 7f03ccd38700 -1 reset not still connected to 0x556b9f278490 2021-02-20T20:47:08.989 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T20:47:08.977+0000 7f03ccd38700 -1 reset not still connected to 0x556b9f279190 2021-02-20T20:47:08.989 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T20:47:08.977+0000 7f03ccd38700 -1 reset not still connected to 0x556bc2b1a4e0 2021-02-20T20:47:08.989 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T20:47:08.977+0000 7f03ccd38700 -1 reset not still connected to 0x556bc2b1a680 2021-02-20T20:47:08.990 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T20:47:08.977+0000 7f03ccd38700 -1 reset not still connected to 0x556bc2b1a820 2021-02-20T20:47:08.990 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T20:47:08.977+0000 7f03ccd38700 -1 reset not still connected to 0x556bc2b1c080 2021-02-20T20:47:08.991 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T20:47:08.977+0000 7f03ccd38700 -1 reset not still connected to 0x556bc2b1d400 2021-02-20T20:47:11.540 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T20:47:11.551 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T20:47:11.552 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T20:47:11.553 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T20:47:11.545+0000 7efccac03700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 66557) UID: 1000 2021-02-20T20:47:11.753 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:47:13.552 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T20:47:13.553 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:47:13.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:47:13.554 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T20:47:13.554 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T20:47:13.554 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T20:47:13.557 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T20:47:13.557 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T20:47:51.670 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T20:47:51.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T20:47:51.681 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T20:47:51.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T20:47:51.676+0000 7f583df2e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 67291) UID: 1000 2021-02-20T20:47:51.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:47:59.682 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T20:48:03.701 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:48:03.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:48:03.710 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T20:48:03.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T20:48:03.711 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T20:48:03.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T20:48:03.721 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T20:48:28.558 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T20:48:28.566 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T20:48:28.567 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T20:48:28.568 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T20:48:28.563+0000 7f090454e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 67936) UID: 1000 2021-02-20T20:48:28.768 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:48:34.567 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T20:48:34.568 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:48:34.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:48:34.569 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T20:48:34.569 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T20:48:34.569 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T20:48:34.572 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T20:48:34.572 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T20:49:07.722 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T20:49:07.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T20:49:07.731 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T20:49:07.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T20:49:07.730+0000 7fd365ddc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 68027) UID: 1000 2021-02-20T20:49:07.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:49:14.731 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T20:49:14.738 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:49:14.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:49:14.739 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T20:49:14.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T20:49:14.740 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T20:49:14.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T20:49:14.743 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T20:50:25.573 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.97472626335445) > weight (0.25) 2021-02-20T20:50:25.585 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2567830189301713) > weight (0.25) 2021-02-20T20:50:25.585 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.657671803181061) > weight (0.25) 2021-02-20T20:50:25.585 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6560098155707171) > weight (0.25) 2021-02-20T20:50:25.585 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T20:51:10.744 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.42266607302843007) > weight (0.25) 2021-02-20T20:51:10.751 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.945393723418103) > weight (0.25) 2021-02-20T20:51:10.751 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.28967464724338243) > weight (0.25) 2021-02-20T20:51:10.751 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T20:51:10.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T20:51:10.753 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T20:51:10.753 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T20:51:10.753 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:51:10.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T20:51:10.747+0000 7f73a15ee700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 67845) UID: 1000 2021-02-20T20:51:10.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:51:16.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:51:19.691 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T20:51:19.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T20:51:19.692 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T20:51:19.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T20:51:19.698 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T20:51:55.587 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T20:51:55.588 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T20:51:55.588 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T20:51:55.590 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T20:51:55.585+0000 7f2581c1b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 68119) UID: 1000 2021-02-20T20:51:55.790 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:52:02.589 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T20:52:02.590 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:52:02.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:52:02.591 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T20:52:02.591 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T20:52:02.591 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T20:52:02.594 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T20:52:02.594 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T20:53:13.699 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.35072985416286817) > weight (0.25) 2021-02-20T20:53:13.708 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8779512342049834) > weight (0.25) 2021-02-20T20:53:13.708 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3808061288263187) > weight (0.25) 2021-02-20T20:53:13.709 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5668147828426926) > weight (0.25) 2021-02-20T20:53:13.709 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T20:53:51.596 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6057750803115913) > weight (0.25) 2021-02-20T20:53:51.604 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.40718116877501354) > weight (0.25) 2021-02-20T20:53:51.604 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.471769229426907) > weight (0.25) 2021-02-20T20:53:51.604 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T20:53:51.605 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T20:53:51.606 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T20:53:51.606 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T20:53:51.602+0000 7fca84853700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 65998) UID: 1000 2021-02-20T20:53:51.807 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:53:58.606 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T20:53:58.730 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:53:58.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:53:58.732 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T20:53:58.732 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T20:53:58.733 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T20:53:58.738 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T20:53:58.738 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T20:54:47.710 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7399379813619161) > weight (0.25) 2021-02-20T20:54:47.717 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2971461881109807) > weight (0.25) 2021-02-20T20:54:47.717 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5770516917821114) > weight (0.25) 2021-02-20T20:54:47.718 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.795711954312307) > weight (0.25) 2021-02-20T20:54:47.718 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T20:55:28.740 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T20:55:28.742 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T20:55:28.742 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T20:55:28.742 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T20:55:28.743 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:55:28.744 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T20:55:28.740+0000 7f796a0ac700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 68395) UID: 1000 2021-02-20T20:55:28.943 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:55:34.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:55:34.845 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T20:55:34.845 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T20:55:34.846 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T20:55:34.848 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T20:55:34.849 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T20:56:23.718 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8712496725830042) > weight (0.25) 2021-02-20T20:56:23.732 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9594995467497929) > weight (0.25) 2021-02-20T20:56:23.732 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T20:56:23.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T20:56:23.733 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T20:56:23.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T20:56:23.730+0000 7f5060411700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 65629) UID: 1000 2021-02-20T20:56:24.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:56:24.734 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T20:56:24.734 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:56:24.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:56:24.735 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T20:56:24.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T20:56:24.736 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T20:56:24.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T20:56:24.747 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T20:56:56.850 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T20:56:56.851 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T20:56:56.851 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T20:56:56.852 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T20:56:56.845+0000 7f20037f9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 68582) UID: 1000 2021-02-20T20:56:57.052 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:56:59.852 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T20:56:59.853 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:56:59.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:56:59.854 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T20:56:59.854 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T20:56:59.854 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T20:56:59.857 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T20:56:59.858 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T20:57:40.748 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T20:57:40.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T20:57:40.756 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T20:57:40.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T20:57:40.752+0000 7f6e9c5dd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 68211) UID: 1000 2021-02-20T20:57:40.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:57:49.756 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T20:57:49.757 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:57:49.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:57:49.758 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T20:57:49.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T20:57:49.758 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T20:57:49.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T20:57:49.761 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T20:58:09.328 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T20:58:09.323+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a6d6c000 2021-02-20T20:58:09.329 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T20:58:09.323+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a6d6e490 2021-02-20T20:58:09.329 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T20:58:09.323+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a6d6eff0 2021-02-20T20:58:09.329 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T20:58:09.323+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6abc0ab10 2021-02-20T20:58:22.859 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T20:58:22.867 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T20:58:22.867 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T20:58:22.868 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T20:58:22.863+0000 7f414db6b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 68773) UID: 1000 2021-02-20T20:58:23.068 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:58:28.868 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T20:58:28.868 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:58:28.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:58:28.869 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T20:58:28.869 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T20:58:28.870 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T20:58:28.872 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T20:58:28.873 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T20:59:27.762 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T20:59:28.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T20:59:28.094 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T20:59:28.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T20:59:28.089+0000 7f8763b97700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 68864) UID: 1000 2021-02-20T20:59:28.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:59:33.095 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T20:59:33.102 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:59:33.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:59:33.103 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T20:59:33.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T20:59:33.104 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T20:59:33.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T20:59:33.107 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T20:59:41.873 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T20:59:41.874 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T20:59:41.874 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T20:59:41.875 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T20:59:41.869+0000 7f1f5dc87700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 68955) UID: 1000 2021-02-20T20:59:42.076 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T20:59:47.876 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T20:59:47.885 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T20:59:47.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T20:59:47.886 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T20:59:47.886 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T20:59:47.886 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T20:59:47.889 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T20:59:47.889 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T20:59:54.784 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T20:59:54.780+0000 7fb8259e4700 -1 reset not still connected to 0x557863b44680 2021-02-20T20:59:54.788 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T20:59:54.784+0000 7fb8259e4700 -1 reset not still connected to 0x557863b46f20 2021-02-20T21:00:40.108 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T21:00:40.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T21:00:40.122 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T21:00:40.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T21:00:40.115+0000 7fdbc1989700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 69047) UID: 1000 2021-02-20T21:00:40.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:00:46.122 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T21:00:46.232 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:00:46.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:00:46.233 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T21:00:46.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T21:00:46.233 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T21:00:46.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T21:00:46.237 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T21:01:38.890 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6069912137560418) > weight (0.25) 2021-02-20T21:01:38.899 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T21:01:38.900 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T21:01:38.901 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T21:01:38.902 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T21:01:38.898+0000 7fed8c7fb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 67567) UID: 1000 2021-02-20T21:01:39.301 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:01:43.902 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T21:01:48.604 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:01:48.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:01:48.613 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T21:01:48.613 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T21:01:48.615 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T21:01:48.620 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T21:01:48.620 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T21:02:43.238 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.739649677836587) > weight (0.25) 2021-02-20T21:02:43.855 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7865028150532309) > weight (0.25) 2021-02-20T21:02:43.856 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4250531264697478) > weight (0.25) 2021-02-20T21:02:43.856 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.2593604004608503) > weight (0.25) 2021-02-20T21:02:43.856 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T21:03:09.621 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5114937507954965) > weight (0.25) 2021-02-20T21:03:09.627 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T21:03:09.628 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T21:03:09.628 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T21:03:09.629 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T21:03:09.623+0000 7fe67bf51700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 69323) UID: 1000 2021-02-20T21:03:09.829 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:03:15.629 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T21:03:15.638 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:03:15.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:03:15.639 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T21:03:15.639 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T21:03:15.640 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T21:03:15.643 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T21:03:15.643 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T21:03:47.857 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T21:03:47.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T21:03:47.865 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T21:03:47.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T21:03:47.862+0000 7f5dfc30e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 69230) UID: 1000 2021-02-20T21:03:48.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:03:52.866 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T21:03:52.867 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:03:52.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:03:52.867 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T21:03:52.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T21:03:52.868 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T21:03:52.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T21:03:52.871 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T21:04:38.644 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T21:04:38.652 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T21:04:38.653 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T21:04:38.653 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T21:04:38.645+0000 7f73c3cf8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 69138) UID: 1000 2021-02-20T21:04:38.854 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:04:44.653 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T21:04:44.654 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:04:44.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:04:44.655 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T21:04:44.655 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T21:04:44.655 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T21:04:44.658 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T21:04:44.658 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T21:05:02.872 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T21:05:02.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T21:05:02.884 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T21:05:02.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T21:05:02.880+0000 7fcd2f37b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 69505) UID: 1000 2021-02-20T21:05:03.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:05:03.885 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T21:05:03.886 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:05:03.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:05:03.887 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T21:05:03.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T21:05:03.887 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T21:05:03.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T21:05:03.890 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T21:06:25.659 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T21:06:25.667 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T21:06:25.667 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T21:06:25.668 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T21:06:25.662+0000 7fe294a31700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 69597) UID: 1000 2021-02-20T21:06:25.869 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:06:26.669 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T21:06:26.669 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:06:26.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:06:26.670 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T21:06:26.670 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T21:06:26.671 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T21:06:26.673 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T21:06:26.674 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T21:07:01.891 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3497325382351376) > weight (0.25) 2021-02-20T21:07:02.148 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8590124030152833) > weight (0.25) 2021-02-20T21:07:02.148 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7144234650733001) > weight (0.25) 2021-02-20T21:07:02.148 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T21:07:02.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T21:07:02.149 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T21:07:02.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T21:07:02.145+0000 7fa43392d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 68304) UID: 1000 2021-02-20T21:07:02.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:07:04.150 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T21:07:04.151 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:07:04.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:07:04.151 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T21:07:04.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T21:07:04.152 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T21:07:04.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T21:07:04.155 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T21:08:22.674 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5061850039694257) > weight (0.25) 2021-02-20T21:08:22.683 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T21:08:22.684 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T21:08:22.684 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T21:08:22.685 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T21:08:22.679+0000 7f749c91d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 69414) UID: 1000 2021-02-20T21:08:22.885 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:08:24.686 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T21:08:24.686 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:08:24.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:08:24.688 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T21:08:24.689 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T21:08:24.689 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T21:08:24.692 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T21:08:24.693 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T21:08:26.156 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9483961096081107) > weight (0.25) 2021-02-20T21:08:26.162 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8544595179509348) > weight (0.25) 2021-02-20T21:08:26.162 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.906725756084123) > weight (0.25) 2021-02-20T21:08:26.163 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9599895919727625) > weight (0.25) 2021-02-20T21:08:26.163 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T21:09:32.165 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3743988455526238) > weight (0.25) 2021-02-20T21:09:33.360 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4577194787749047) > weight (0.25) 2021-02-20T21:09:33.360 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4065853843253143) > weight (0.25) 2021-02-20T21:09:33.361 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.815013338857604) > weight (0.25) 2021-02-20T21:09:33.361 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T21:09:37.694 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9046156765158624) > weight (0.25) 2021-02-20T21:09:37.694 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T21:09:37.695 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T21:09:37.695 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T21:09:37.696 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T21:09:37.693+0000 7fa6b2cb3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 69974) UID: 1000 2021-02-20T21:09:37.897 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:09:44.697 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T21:09:45.117 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:09:45.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:09:45.118 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T21:09:45.119 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T21:09:45.119 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T21:09:45.122 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T21:09:45.122 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T21:11:04.123 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T21:11:04.144 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T21:11:04.145 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T21:11:04.148 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T21:11:04.143+0000 7f4a0dd19700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 69783) UID: 1000 2021-02-20T21:11:04.348 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:11:08.146 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T21:11:08.146 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:11:08.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:11:08.147 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T21:11:08.147 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T21:11:08.148 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T21:11:08.150 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T21:11:08.151 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T21:11:26.362 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T21:11:26.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T21:11:26.718 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T21:11:26.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T21:11:26.714+0000 7feb42532700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 69691) UID: 1000 2021-02-20T21:11:26.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:11:28.719 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T21:11:28.720 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:11:28.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:11:28.720 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T21:11:28.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T21:11:28.721 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T21:11:28.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T21:11:28.730 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T21:12:09.151 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7192966629261852) > weight (0.25) 2021-02-20T21:12:09.161 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7476845372776149) > weight (0.25) 2021-02-20T21:12:09.161 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.32868538938402214) > weight (0.25) 2021-02-20T21:12:09.161 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T21:12:09.162 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T21:12:09.162 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T21:12:09.163 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T21:12:09.161+0000 7f550e2c3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 68487) UID: 1000 2021-02-20T21:12:09.364 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:12:18.164 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T21:12:18.164 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:12:18.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:12:18.166 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T21:12:18.166 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T21:12:18.167 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T21:12:18.170 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T21:12:18.170 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T21:12:43.372 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T21:12:43.360+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a600e7d0 2021-02-20T21:12:58.731 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T21:12:58.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T21:12:58.740 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T21:12:58.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T21:12:58.736+0000 7f45c04a0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 70249) UID: 1000 2021-02-20T21:12:58.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:13:02.741 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T21:13:02.741 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:13:02.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:13:02.742 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T21:13:02.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T21:13:02.743 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T21:13:02.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T21:13:02.746 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T21:14:14.171 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7755761703359824) > weight (0.25) 2021-02-20T21:14:14.387 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T21:14:14.388 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T21:14:14.390 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T21:14:14.391 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T21:14:14.382+0000 7f7ae41f2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 70066) UID: 1000 2021-02-20T21:14:14.590 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:14:16.391 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T21:14:16.391 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:14:16.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:14:16.392 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T21:14:16.392 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T21:14:16.393 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T21:14:16.395 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T21:14:16.396 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T21:14:40.748 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9835453237469917) > weight (0.25) 2021-02-20T21:14:40.748 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T21:14:40.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T21:14:40.749 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T21:14:40.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T21:14:40.745+0000 7fc8a165a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 67109) UID: 1000 2021-02-20T21:14:40.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:14:43.750 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T21:14:43.751 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:14:43.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:14:43.752 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T21:14:43.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T21:14:43.753 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T21:14:43.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T21:14:43.757 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T21:15:58.759 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.41585440424486664) > weight (0.25) 2021-02-20T21:15:58.876 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4555235600148593) > weight (0.25) 2021-02-20T21:15:58.877 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3725342472278067) > weight (0.25) 2021-02-20T21:15:58.877 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T21:15:58.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T21:15:58.878 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T21:15:58.878 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T21:15:58.879 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:15:58.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T21:15:58.875+0000 7f5bea04f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 69875) UID: 1000 2021-02-20T21:15:59.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:16:02.397 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9763995289468773) > weight (0.25) 2021-02-20T21:16:02.397 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T21:16:02.398 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T21:16:02.398 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T21:16:02.399 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T21:16:02.395+0000 7f6ea5842700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 70525) UID: 1000 2021-02-20T21:16:02.599 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:16:04.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:16:04.982 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T21:16:04.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T21:16:04.983 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T21:16:04.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T21:16:04.986 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T21:16:09.399 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T21:16:09.399 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:16:09.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:16:09.400 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T21:16:09.400 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T21:16:09.401 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T21:16:09.403 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T21:16:09.404 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T21:17:17.405 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T21:17:17.414 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T21:17:17.415 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T21:17:17.416 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T21:17:17.413+0000 7fe962e7b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 70158) UID: 1000 2021-02-20T21:17:17.617 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:17:24.416 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T21:17:24.416 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:17:24.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:17:24.417 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T21:17:24.418 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T21:17:24.418 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T21:17:24.421 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T21:17:24.421 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T21:17:41.987 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4055374427166669) > weight (0.25) 2021-02-20T21:17:41.996 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9067179471761753) > weight (0.25) 2021-02-20T21:17:41.997 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7076229985819271) > weight (0.25) 2021-02-20T21:17:41.997 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.557497618465525) > weight (0.25) 2021-02-20T21:17:41.997 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T21:19:12.422 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8312531273208946) > weight (0.25) 2021-02-20T21:19:12.430 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3737136583682782) > weight (0.25) 2021-02-20T21:19:12.430 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9121568988752616) > weight (0.25) 2021-02-20T21:19:12.430 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5077441497383957) > weight (0.25) 2021-02-20T21:19:12.430 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T21:19:26.998 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7822582621463658) > weight (0.25) 2021-02-20T21:19:32.257 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.51067714524923) > weight (0.25) 2021-02-20T21:19:32.265 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T21:19:32.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T21:19:32.272 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T21:19:32.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T21:19:32.277+0000 7f6db802a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 68674) UID: 1000 2021-02-20T21:19:32.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:19:35.272 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T21:19:35.273 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:19:35.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:19:35.274 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T21:19:35.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T21:19:35.274 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T21:19:35.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T21:19:35.277 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T21:19:36.897 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T21:19:36.888+0000 7fb8259e4700 -1 reset not still connected to 0x5578582496c0 2021-02-20T21:19:36.897 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T21:19:36.888+0000 7fb8259e4700 -1 reset not still connected to 0x557863b45930 2021-02-20T21:19:36.897 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T21:19:36.888+0000 7fb8259e4700 -1 reset not still connected to 0x557866b05ad0 2021-02-20T21:20:41.432 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T21:20:44.647 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T21:20:44.648 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T21:20:44.649 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T21:20:44.643+0000 7f975f7b2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 70898) UID: 1000 2021-02-20T21:20:44.849 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:20:47.649 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T21:20:47.649 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:20:47.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:20:47.650 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T21:20:47.650 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T21:20:47.651 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T21:20:47.653 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T21:20:47.654 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T21:21:13.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8102326869025843) > weight (0.25) 2021-02-20T21:21:13.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.35057632859081433) > weight (0.25) 2021-02-20T21:21:13.279 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T21:21:13.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T21:21:13.281 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T21:21:13.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T21:21:13.275+0000 7fc7fcc06700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 70989) UID: 1000 2021-02-20T21:21:13.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:21:14.281 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T21:21:14.282 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:21:14.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:21:14.282 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T21:21:14.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T21:21:14.283 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T21:21:14.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T21:21:14.286 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T21:22:05.844 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T21:22:05.836+0000 7fb8259e4700 -1 reset not still connected to 0x557858248dd0 2021-02-20T21:22:05.845 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T21:22:05.836+0000 7fb8259e4700 -1 reset not still connected to 0x557863b44a90 2021-02-20T21:22:05.845 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T21:22:05.836+0000 7fb8259e4700 -1 reset not still connected to 0x557863b46be0 2021-02-20T21:22:23.655 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8300513028491977) > weight (0.25) 2021-02-20T21:22:23.663 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2719367539421802) > weight (0.25) 2021-02-20T21:22:23.663 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3438286481717058) > weight (0.25) 2021-02-20T21:22:23.664 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6757584720333437) > weight (0.25) 2021-02-20T21:22:23.664 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T21:22:43.287 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4825859843311956) > weight (0.25) 2021-02-20T21:22:48.865 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.55829158996074) > weight (0.25) 2021-02-20T21:22:48.877 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2809695937493667) > weight (0.25) 2021-02-20T21:22:48.878 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3891775037054567) > weight (0.25) 2021-02-20T21:22:48.878 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T21:23:23.665 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T21:23:23.674 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T21:23:23.674 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T21:23:23.680 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T21:23:23.675+0000 7f3ee66b6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 71090) UID: 1000 2021-02-20T21:23:23.881 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:23:32.676 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T21:23:32.677 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:23:32.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:23:32.678 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T21:23:32.678 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T21:23:32.678 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T21:23:32.681 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T21:23:32.682 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T21:24:46.879 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T21:24:46.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T21:24:46.888 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T21:24:46.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T21:24:46.885+0000 7f5aa4571700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 70434) UID: 1000 2021-02-20T21:24:47.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:24:50.889 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T21:24:50.890 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:24:50.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:24:50.891 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T21:24:50.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T21:24:50.893 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T21:24:50.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T21:24:50.897 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T21:25:06.683 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T21:25:06.684 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T21:25:06.685 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T21:25:06.686 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T21:25:06.680+0000 7f301adbe700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 71274) UID: 1000 2021-02-20T21:25:06.886 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:25:15.685 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T21:25:15.692 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:25:15.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:25:15.693 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T21:25:15.693 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T21:25:15.694 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T21:25:15.696 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T21:25:15.697 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T21:26:07.898 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4782971016125731) > weight (0.25) 2021-02-20T21:26:07.904 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5578200665372585) > weight (0.25) 2021-02-20T21:26:07.905 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3974569120904716) > weight (0.25) 2021-02-20T21:26:07.905 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9265601808269842) > weight (0.25) 2021-02-20T21:26:07.905 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T21:26:54.698 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2972420575563979) > weight (0.25) 2021-02-20T21:26:54.705 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T21:26:54.706 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T21:26:54.706 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T21:26:54.707 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T21:26:54.702+0000 7f5b10099700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 70802) UID: 1000 2021-02-20T21:26:54.908 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:26:58.707 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T21:26:58.712 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:26:58.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:26:58.713 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T21:26:58.713 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T21:26:58.714 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T21:26:58.716 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T21:26:58.717 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T21:27:46.906 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3373770559546658) > weight (0.25) 2021-02-20T21:27:46.914 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T21:27:46.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T21:27:46.915 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T21:27:46.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T21:27:46.912+0000 7fcd84af6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 70616) UID: 1000 2021-02-20T21:27:47.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:27:53.916 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T21:27:53.917 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:27:53.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:27:53.918 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T21:27:53.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T21:27:53.919 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T21:27:53.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T21:27:53.922 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T21:28:03.718 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T21:28:03.725 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T21:28:03.726 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T21:28:03.727 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T21:28:03.724+0000 7f262875a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 71460) UID: 1000 2021-02-20T21:28:03.927 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:28:11.727 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T21:28:11.753 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:28:11.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:28:11.754 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T21:28:11.754 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T21:28:11.755 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T21:28:11.758 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T21:28:11.758 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T21:28:35.803 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T21:28:35.795+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b6c87d40 2021-02-20T21:28:35.803 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T21:28:35.795+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ca07e750 2021-02-20T21:29:05.811 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T21:29:05.802+0000 7f4a1bb72700 -1 reset not still connected to 0x55fc67565450 2021-02-20T21:29:05.811 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T21:29:05.802+0000 7f4a1bb72700 -1 reset not still connected to 0x55fc67566d80 2021-02-20T21:29:05.812 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T21:29:05.806+0000 7f4a1bb72700 -1 reset not still connected to 0x55fc92d74700 2021-02-20T21:29:05.812 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T21:29:05.806+0000 7f4a1bb72700 -1 reset not still connected to 0x55fca2e108a0 2021-02-20T21:29:18.923 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8527289147760737) > weight (0.25) 2021-02-20T21:29:18.931 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.869078563765422) > weight (0.25) 2021-02-20T21:29:18.932 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7327269144234565) > weight (0.25) 2021-02-20T21:29:18.932 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4756360009237379) > weight (0.25) 2021-02-20T21:29:18.932 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T21:29:56.759 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T21:29:56.768 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T21:29:56.769 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T21:29:56.769 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T21:29:56.769 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:29:56.770 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T21:29:56.765+0000 7fad5e70f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 71741) UID: 1000 2021-02-20T21:29:56.970 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:30:02.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:30:02.872 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T21:30:02.872 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T21:30:02.872 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T21:30:02.876 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T21:30:02.876 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T21:30:44.933 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4935363215663471) > weight (0.25) 2021-02-20T21:30:44.942 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5044286993112047) > weight (0.25) 2021-02-20T21:30:44.943 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5618182820920513) > weight (0.25) 2021-02-20T21:30:44.943 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T21:30:44.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T21:30:44.944 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T21:30:44.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T21:30:44.939+0000 7ff28798c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 70711) UID: 1000 2021-02-20T21:30:45.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:30:46.945 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T21:30:46.946 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:30:46.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:30:46.946 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T21:30:46.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T21:30:46.947 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T21:30:46.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T21:30:46.950 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T21:31:12.877 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.410889992520197) > weight (0.25) 2021-02-20T21:31:12.885 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.403028410757571) > weight (0.25) 2021-02-20T21:31:12.885 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T21:31:12.886 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T21:31:12.886 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T21:31:12.887 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T21:31:12.883+0000 7fe80bc9d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 67754) UID: 1000 2021-02-20T21:31:13.288 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:31:19.888 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T21:31:19.902 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:31:19.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:31:19.903 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T21:31:19.903 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T21:31:19.911 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T21:31:19.914 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T21:31:19.914 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T21:32:36.951 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.969580970498865) > weight (0.25) 2021-02-20T21:32:36.984 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7287488472577184) > weight (0.25) 2021-02-20T21:32:36.984 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.618223102861778) > weight (0.25) 2021-02-20T21:32:36.985 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.600800024163092) > weight (0.25) 2021-02-20T21:32:36.985 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T21:33:09.915 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8503681641383452) > weight (0.25) 2021-02-20T21:33:09.924 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T21:33:09.925 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T21:33:09.925 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T21:33:09.926 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T21:33:09.924+0000 7f3597be3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 71551) UID: 1000 2021-02-20T21:33:10.126 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:33:18.926 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T21:33:18.927 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:33:18.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:33:18.928 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T21:33:18.929 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T21:33:18.929 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T21:33:18.933 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T21:33:18.934 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T21:33:51.986 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6408334488669362) > weight (0.25) 2021-02-20T21:33:51.993 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7531006749776975) > weight (0.25) 2021-02-20T21:33:51.993 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9198587078711146) > weight (0.25) 2021-02-20T21:33:51.993 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T21:33:51.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T21:33:51.994 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T21:33:51.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T21:33:51.990+0000 7f8428afe700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 71927) UID: 1000 2021-02-20T21:33:52.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:33:55.994 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T21:33:55.995 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:33:55.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:33:55.996 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T21:33:55.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T21:33:55.997 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T21:33:55.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T21:33:56.000 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T21:34:35.935 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5802998961804774) > weight (0.25) 2021-02-20T21:34:35.957 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6956544393842463) > weight (0.25) 2021-02-20T21:34:35.957 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.30586002218588637) > weight (0.25) 2021-02-20T21:34:35.957 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8198702686277549) > weight (0.25) 2021-02-20T21:34:35.958 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T21:35:05.001 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5408682057476729) > weight (0.25) 2021-02-20T21:35:05.002 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.37169216166924535) > weight (0.25) 2021-02-20T21:35:05.002 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.49671467230278177) > weight (0.25) 2021-02-20T21:35:05.002 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T21:35:05.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T21:35:05.003 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T21:35:05.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T21:35:05.001+0000 7fe54a6b9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 72201) UID: 1000 2021-02-20T21:35:05.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:35:06.005 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T21:35:06.005 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:35:06.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:35:06.006 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T21:35:06.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T21:35:06.006 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T21:35:06.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T21:35:06.009 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T21:36:03.958 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T21:36:03.967 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T21:36:03.967 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T21:36:03.968 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T21:36:03.963+0000 7f0e53d8c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 71835) UID: 1000 2021-02-20T21:36:04.168 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:36:07.969 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T21:36:07.974 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:36:07.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:36:07.976 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T21:36:07.976 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T21:36:07.977 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T21:36:07.988 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T21:36:07.989 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T21:36:26.010 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.36875506503339295) > weight (0.25) 2021-02-20T21:36:26.027 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4576518902114739) > weight (0.25) 2021-02-20T21:36:26.027 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7582167201303113) > weight (0.25) 2021-02-20T21:36:26.028 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T21:36:26.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T21:36:26.030 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T21:36:26.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T21:36:26.022+0000 7f1625df7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 72295) UID: 1000 2021-02-20T21:36:26.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:36:32.032 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T21:36:32.039 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:36:32.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:36:32.040 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T21:36:32.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T21:36:32.042 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T21:36:32.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T21:36:32.045 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T21:37:29.990 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9165648032456521) > weight (0.25) 2021-02-20T21:37:29.997 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T21:37:29.998 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T21:37:29.998 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T21:37:29.999 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T21:37:29.997+0000 7fbbb418c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 72109) UID: 1000 2021-02-20T21:37:30.199 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:37:33.000 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T21:37:33.000 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:37:33.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:37:33.001 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T21:37:33.001 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T21:37:33.001 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T21:37:33.004 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T21:37:33.004 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T21:37:38.275 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T21:37:38.273+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ca0802f0 2021-02-20T21:37:57.046 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7660441374797662) > weight (0.25) 2021-02-20T21:37:57.046 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7591757218111185) > weight (0.25) 2021-02-20T21:37:57.047 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8987981350390516) > weight (0.25) 2021-02-20T21:37:57.047 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6866037785697184) > weight (0.25) 2021-02-20T21:37:57.047 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T21:38:42.006 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3307663372973709) > weight (0.25) 2021-02-20T21:38:42.013 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.27868681084985847) > weight (0.25) 2021-02-20T21:38:42.013 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7869582762686692) > weight (0.25) 2021-02-20T21:38:42.014 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.37743388256374455) > weight (0.25) 2021-02-20T21:38:42.014 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T21:39:45.047 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8967970999395504) > weight (0.25) 2021-02-20T21:39:45.089 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T21:39:45.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T21:39:45.091 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T21:39:45.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T21:39:45.085+0000 7fcade5f1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 71650) UID: 1000 2021-02-20T21:39:45.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:39:48.093 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T21:39:48.100 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:39:48.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:39:48.101 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T21:39:48.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T21:39:48.101 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T21:39:48.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T21:39:48.104 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T21:40:37.014 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8157340861411949) > weight (0.25) 2021-02-20T21:40:37.027 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4669662724169782) > weight (0.25) 2021-02-20T21:40:37.027 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4542164689330256) > weight (0.25) 2021-02-20T21:40:37.027 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.40172339930518386) > weight (0.25) 2021-02-20T21:40:37.028 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T21:41:25.105 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7202588317838163) > weight (0.25) 2021-02-20T21:41:26.330 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4449217743397538) > weight (0.25) 2021-02-20T21:41:26.332 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5133334749065106) > weight (0.25) 2021-02-20T21:41:26.333 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7388896952952997) > weight (0.25) 2021-02-20T21:41:26.333 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T21:41:47.028 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8172691783505815) > weight (0.25) 2021-02-20T21:41:47.029 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9057262304072407) > weight (0.25) 2021-02-20T21:41:47.029 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9849121647471004) > weight (0.25) 2021-02-20T21:41:47.029 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9496587348498885) > weight (0.25) 2021-02-20T21:41:47.029 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T21:43:00.334 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.41049710436804254) > weight (0.25) 2021-02-20T21:43:00.342 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7931433467136046) > weight (0.25) 2021-02-20T21:43:00.343 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5299087568040706) > weight (0.25) 2021-02-20T21:43:00.343 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8324729994106845) > weight (0.25) 2021-02-20T21:43:00.343 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T21:43:17.030 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8461879087957299) > weight (0.25) 2021-02-20T21:43:17.063 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T21:43:17.064 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T21:43:17.064 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T21:43:17.065 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T21:43:17.060+0000 7f1aa6b30700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 72569) UID: 1000 2021-02-20T21:43:17.266 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:43:18.065 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T21:43:18.066 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:43:18.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:43:18.067 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T21:43:18.068 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T21:43:18.069 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T21:43:18.071 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T21:43:18.072 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T21:44:39.073 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T21:44:39.082 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T21:44:39.082 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T21:44:39.083 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T21:44:39.077+0000 7fe8ebb92700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 72387) UID: 1000 2021-02-20T21:44:39.284 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:44:41.344 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6607030902356443) > weight (0.25) 2021-02-20T21:44:41.344 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3841585148171669) > weight (0.25) 2021-02-20T21:44:41.344 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.262663518359016) > weight (0.25) 2021-02-20T21:44:41.345 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.37094056186871693) > weight (0.25) 2021-02-20T21:44:41.345 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T21:44:42.083 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T21:44:43.091 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:44:43.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:44:43.092 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T21:44:43.093 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T21:44:43.093 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T21:44:43.101 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T21:44:43.101 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T21:46:16.102 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3081415205842687) > weight (0.25) 2021-02-20T21:46:16.111 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9121858530801948) > weight (0.25) 2021-02-20T21:46:16.112 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6170516855865423) > weight (0.25) 2021-02-20T21:46:16.112 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.43590796636921214) > weight (0.25) 2021-02-20T21:46:16.112 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T21:46:26.345 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T21:46:26.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T21:46:26.349 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T21:46:26.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T21:46:26.347+0000 7fe16c171700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 71365) UID: 1000 2021-02-20T21:46:26.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:46:27.350 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T21:46:27.350 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:46:27.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:46:27.351 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T21:46:27.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T21:46:27.352 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T21:46:27.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T21:46:27.355 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T21:47:32.355 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T21:47:32.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T21:47:32.365 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T21:47:32.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T21:47:32.361+0000 7f0aea787700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 72942) UID: 1000 2021-02-20T21:47:32.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:47:39.365 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T21:47:39.396 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:47:39.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:47:39.397 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T21:47:39.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T21:47:39.398 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T21:47:39.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T21:47:39.403 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T21:48:03.114 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4151739475391135) > weight (0.25) 2021-02-20T21:48:03.121 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9548743780649817) > weight (0.25) 2021-02-20T21:48:03.122 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.527258738659132) > weight (0.25) 2021-02-20T21:48:03.122 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5297656292395372) > weight (0.25) 2021-02-20T21:48:03.123 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T21:49:07.404 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T21:49:07.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T21:49:07.414 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T21:49:07.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T21:49:07.410+0000 7f1cf66c9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 73041) UID: 1000 2021-02-20T21:49:07.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:49:08.415 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T21:49:08.416 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:49:08.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:49:08.416 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T21:49:08.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T21:49:08.417 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T21:49:08.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T21:49:08.420 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T21:49:36.124 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T21:49:36.130 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T21:49:36.131 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T21:49:36.132 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T21:49:36.130+0000 7f48e4862700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 72846) UID: 1000 2021-02-20T21:49:36.332 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:49:45.132 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T21:49:45.133 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:49:45.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:49:45.134 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T21:49:45.134 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T21:49:45.134 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T21:49:45.137 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T21:49:45.137 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T21:50:09.421 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9560068352960713) > weight (0.25) 2021-02-20T21:50:09.433 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.46119808076545377) > weight (0.25) 2021-02-20T21:50:09.433 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T21:50:09.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T21:50:09.435 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T21:50:09.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T21:50:09.433+0000 7f6ac06ef700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 71181) UID: 1000 2021-02-20T21:50:09.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:50:12.437 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T21:50:14.214 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:50:14.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:50:14.215 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T21:50:14.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T21:50:14.215 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T21:50:14.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T21:50:14.220 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T21:50:52.138 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.38927251891691605) > weight (0.25) 2021-02-20T21:50:52.139 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T21:50:52.139 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T21:50:52.140 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T21:50:52.140 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T21:50:52.136+0000 7ff5a7f09700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 72754) UID: 1000 2021-02-20T21:50:52.341 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:50:58.141 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T21:50:58.141 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:50:58.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:50:58.142 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T21:50:58.142 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T21:50:58.143 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T21:50:58.147 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T21:50:58.147 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T21:51:55.221 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5107203199183769) > weight (0.25) 2021-02-20T21:51:55.251 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T21:51:55.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T21:51:55.252 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T21:51:55.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T21:51:55.246+0000 7f7646fbb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 72661) UID: 1000 2021-02-20T21:51:55.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:52:02.254 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T21:52:02.262 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:52:02.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:52:02.263 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T21:52:02.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T21:52:02.264 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T21:52:02.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T21:52:02.267 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T21:52:06.148 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.766805155520284) > weight (0.25) 2021-02-20T21:52:06.148 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3687155576179969) > weight (0.25) 2021-02-20T21:52:06.148 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T21:52:06.149 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T21:52:06.149 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T21:52:06.149 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T21:52:06.150 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:52:06.150 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T21:52:06.146+0000 7f34bc6f1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 72018) UID: 1000 2021-02-20T21:52:06.350 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:52:12.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:52:12.251 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T21:52:12.252 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T21:52:12.252 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T21:52:12.254 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T21:52:12.255 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T21:53:17.256 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5169106519166424) > weight (0.25) 2021-02-20T21:53:17.305 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.25984173840200175) > weight (0.25) 2021-02-20T21:53:17.306 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9516430707747766) > weight (0.25) 2021-02-20T21:53:17.306 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.32371371875711763) > weight (0.25) 2021-02-20T21:53:17.306 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T21:53:38.268 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8621430272529045) > weight (0.25) 2021-02-20T21:53:38.276 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3320459483478183) > weight (0.25) 2021-02-20T21:53:38.276 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6856672097966823) > weight (0.25) 2021-02-20T21:53:38.276 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9402561898545136) > weight (0.25) 2021-02-20T21:53:38.277 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T21:54:48.444 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T21:54:48.437+0000 7f03ccd38700 -1 reset not still connected to 0x556b8baa9040 2021-02-20T21:54:54.278 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6766211395118745) > weight (0.25) 2021-02-20T21:54:54.287 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.788689745297549) > weight (0.25) 2021-02-20T21:54:54.287 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8776420935721411) > weight (0.25) 2021-02-20T21:54:54.287 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6886565516300379) > weight (0.25) 2021-02-20T21:54:54.288 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T21:55:06.307 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.35989499349998144) > weight (0.25) 2021-02-20T21:55:06.318 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.25328927807386137) > weight (0.25) 2021-02-20T21:55:06.318 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.644006086056471) > weight (0.25) 2021-02-20T21:55:06.318 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7836026551874437) > weight (0.25) 2021-02-20T21:55:06.319 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T21:56:14.289 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9552353805451128) > weight (0.25) 2021-02-20T21:56:14.300 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T21:56:14.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T21:56:14.301 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T21:56:14.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T21:56:14.295+0000 7fbdd7eea700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 73507) UID: 1000 2021-02-20T21:56:14.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:56:15.303 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T21:56:15.303 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:56:15.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:56:15.304 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T21:56:15.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T21:56:15.305 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T21:56:15.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T21:56:15.320 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T21:56:39.320 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5181621575336474) > weight (0.25) 2021-02-20T21:56:39.382 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T21:56:39.383 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T21:56:39.384 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T21:56:39.385 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T21:56:39.378+0000 7f28f39ff700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 73416) UID: 1000 2021-02-20T21:56:39.585 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:56:40.392 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T21:56:40.382+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a3c7c680 2021-02-20T21:56:40.395 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T21:56:40.386+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a3c7fc20 2021-02-20T21:56:40.396 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T21:56:40.386+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b5aa2f70 2021-02-20T21:56:42.384 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T21:56:42.389 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:56:42.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:56:42.391 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T21:56:42.391 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T21:56:42.392 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T21:56:42.397 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T21:56:42.398 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T21:57:53.321 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.788048207456739) > weight (0.25) 2021-02-20T21:57:53.332 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.38791651956510265) > weight (0.25) 2021-02-20T21:57:53.332 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T21:57:53.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T21:57:53.333 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T21:57:53.334 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T21:57:53.334 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:57:53.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T21:57:53.328+0000 7faaddd57700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 73324) UID: 1000 2021-02-20T21:57:53.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:57:59.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:57:59.443 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T21:57:59.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T21:57:59.444 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T21:57:59.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T21:57:59.453 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T21:58:00.400 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9150625551070378) > weight (0.25) 2021-02-20T21:58:00.401 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T21:58:00.401 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T21:58:00.402 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T21:58:00.404 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T21:58:00.400+0000 7fa13bf0e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 73785) UID: 1000 2021-02-20T21:58:00.605 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:58:05.404 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T21:58:05.405 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:58:05.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:58:05.407 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T21:58:05.407 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T21:58:05.408 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T21:58:05.411 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T21:58:05.411 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T21:58:18.477 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T21:58:18.473+0000 7fb8259e4700 -1 reset not still connected to 0x557866b06a40 2021-02-20T21:59:21.412 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.25727009948229285) > weight (0.25) 2021-02-20T21:59:21.436 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T21:59:21.437 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T21:59:21.437 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T21:59:21.438 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T21:59:21.434+0000 7f28843e7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 73967) UID: 1000 2021-02-20T21:59:21.639 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:59:30.439 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T21:59:30.447 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:59:30.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:59:30.449 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T21:59:30.449 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T21:59:30.450 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T21:59:30.468 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T21:59:30.469 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T21:59:52.454 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5166910200362876) > weight (0.25) 2021-02-20T21:59:52.489 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.38229085510445626) > weight (0.25) 2021-02-20T21:59:52.490 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T21:59:52.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T21:59:52.491 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T21:59:52.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T21:59:52.489+0000 7f781e0ff700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 73876) UID: 1000 2021-02-20T21:59:52.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T21:59:55.492 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T21:59:55.498 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T21:59:55.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T21:59:55.498 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T21:59:55.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T21:59:55.499 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T21:59:55.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T21:59:55.502 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T22:00:39.470 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8378658353803544) > weight (0.25) 2021-02-20T22:00:39.479 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T22:00:39.480 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T22:00:39.480 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T22:00:39.482 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T22:00:39.476+0000 7f73f0d52700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 74059) UID: 1000 2021-02-20T22:00:39.682 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:00:43.481 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T22:00:43.504 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:00:43.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:00:43.505 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T22:00:43.505 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T22:00:43.507 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T22:00:43.509 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T22:00:43.510 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T22:01:14.128 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:01:14.117+0000 7fb8259e4700 -1 reset not still connected to 0x557848e71e10 2021-02-20T22:01:14.130 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:01:14.125+0000 7fb8259e4700 -1 reset not still connected to 0x55785824b400 2021-02-20T22:01:14.132 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:01:14.125+0000 7fb8259e4700 -1 reset not still connected to 0x55785f2ad8e0 2021-02-20T22:01:47.503 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6920501423946166) > weight (0.25) 2021-02-20T22:01:47.515 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T22:01:47.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T22:01:47.517 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T22:01:47.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T22:01:47.514+0000 7fd515119700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 73694) UID: 1000 2021-02-20T22:01:47.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:01:50.511 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5787640549517699) > weight (0.25) 2021-02-20T22:01:50.512 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T22:01:50.513 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T22:01:50.513 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T22:01:50.514 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T22:01:50.514 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:01:50.515 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T22:01:50.506+0000 7f470706e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 74309) UID: 1000 2021-02-20T22:01:50.518 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T22:01:50.519 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:01:50.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:01:50.521 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T22:01:50.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T22:01:50.522 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T22:01:50.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T22:01:50.526 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T22:01:50.714 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:01:56.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:01:56.616 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T22:01:56.617 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T22:01:56.617 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T22:01:56.625 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T22:01:56.626 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T22:02:56.626 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7743643564002071) > weight (0.25) 2021-02-20T22:02:56.637 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.46694663617937937) > weight (0.25) 2021-02-20T22:02:56.637 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T22:02:56.638 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T22:02:56.638 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T22:02:56.639 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T22:02:56.636+0000 7f8511e98700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 73598) UID: 1000 2021-02-20T22:02:56.840 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:03:04.640 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T22:03:04.647 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:03:04.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:03:04.648 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T22:03:04.649 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T22:03:04.650 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T22:03:04.673 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T22:03:04.674 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T22:03:11.528 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.552739602139483) > weight (0.25) 2021-02-20T22:03:11.534 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T22:03:11.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T22:03:11.535 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T22:03:11.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T22:03:11.532+0000 7f2892471700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 74402) UID: 1000 2021-02-20T22:03:11.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:03:16.536 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T22:03:16.537 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:03:16.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:03:16.539 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T22:03:16.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T22:03:16.540 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T22:03:16.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T22:03:16.545 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T22:04:17.675 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2530322761391999) > weight (0.25) 2021-02-20T22:04:17.684 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.30227843647432995) > weight (0.25) 2021-02-20T22:04:17.684 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T22:04:17.685 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T22:04:17.686 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T22:04:17.687 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T22:04:17.678+0000 7f6eab248700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 74584) UID: 1000 2021-02-20T22:04:17.887 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:04:22.688 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T22:04:22.694 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:04:22.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:04:22.695 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T22:04:22.696 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T22:04:22.696 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T22:04:22.700 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T22:04:22.701 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T22:04:53.546 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T22:04:53.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T22:04:53.574 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T22:04:53.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T22:04:53.569+0000 7f8a773d6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 73132) UID: 1000 2021-02-20T22:04:53.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:04:57.886 INFO:tasks.ceph.cluster1.osd.0.smithi170.stderr:2021-02-20T22:04:57.879+0000 7f184c55a700 -1 reset not still connected to 0x565513719ba0 2021-02-20T22:04:57.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636fba97680 [3/2464eeeb-6954-426d-a8e5-129943cea55c] start: already running 2021-02-20T22:04:57.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636fba97900 [3/3ab5666f-763c-442d-bc17-ea037dcc3a42] start: already running 2021-02-20T22:04:57.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636fba97b80 [3/3c6940ee-a642-422e-aafb-e3920bba87cf] start: already running 2021-02-20T22:04:57.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636fba97e00 [3/3d38caf8-6420-4f62-95dc-628caa5144da] start: already running 2021-02-20T22:04:57.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636fba98080 [3/49a0b5eb-7d69-4824-806b-1ece12354eb9] start: already running 2021-02-20T22:04:57.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffcea800 [3/bbf1cffa-5e93-4c1c-937d-71c8c54806ff] start: already running 2021-02-20T22:04:57.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffceaa80 [3/cdea041d-e1fe-4928-9842-3ce9553bc980] start: already running 2021-02-20T22:04:57.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffcead00 [3/cdfd4b53-cfc9-488d-957e-07aa3d636acf] start: already running 2021-02-20T22:04:57.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffceaf80 [3/ce32d773-a477-4402-af84-407be17c35f5] start: already running 2021-02-20T22:04:57.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffceb200 [3/dd0c591d-efef-4cf8-9976-7731785fbcd4] start: already running 2021-02-20T22:04:57.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffceb480 [3/dd0f679e-1316-4b67-b5c5-1d6037eb2c90] start: already running 2021-02-20T22:04:57.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffceb700 [3/def81481-a32e-45b7-8067-6c32798fc165] start: already running 2021-02-20T22:04:57.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffceb980 [3/e5f43e4f-211a-45bf-bf8b-2e986b46e688] start: already running 2021-02-20T22:04:57.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffcebc00 [3/eafa574f-c2c8-4659-b559-be75ec7d56bb] start: already running 2021-02-20T22:04:57.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffd1a000 [3/ef164021-65e4-434b-a5d7-bcdd53df22fd] start: already running 2021-02-20T22:04:57.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffd1a280 [3/ff38d3a1-22f6-4a7c-a814-2d4505cf6570] start: already running 2021-02-20T22:04:57.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffd1a500 [3/ff64bb69-23c4-4b3a-9b65-a48c78854429] start: already running 2021-02-20T22:04:57.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636fba98300 [3/4c862af6-d1da-4215-b6b8-461f3883b104] start: already running 2021-02-20T22:04:57.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636fba98580 [3/4cba354e-99d2-4859-a4d0-baeec2203656] start: already running 2021-02-20T22:04:57.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636fba98800 [3/567ace7d-44e4-4d42-b47a-38bd35ec0e11] start: already running 2021-02-20T22:04:57.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636fba98a80 [3/5786ec48-0da5-470d-b6a6-ddd71a666657] start: already running 2021-02-20T22:04:57.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636fba98d00 [3/5c036645-b242-4fa8-bde0-d5b28dda3fd3] start: already running 2021-02-20T22:04:57.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66b46700 -1 rbd::mirror::ImageReplayer: 0x5636fba98f80 [3/5f3204b5-90c6-4244-a940-e749860457d5] start: already running 2021-02-20T22:04:57.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636fba99200 [3/675a1b7b-4e17-4b1e-9e92-eaf406f1c33f] start: already running 2021-02-20T22:04:57.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636fba99480 [3/681dbea5-7557-4dba-a92d-01da85613e31] start: already running 2021-02-20T22:04:57.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636fba99700 [3/753c14e7-97d4-4109-8814-2c7f1abbbee8] start: already running 2021-02-20T22:04:57.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636fba99980 [3/7658e9e0-5116-4c47-bad5-4e7894e27da7] start: already running 2021-02-20T22:04:57.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66b46700 -1 rbd::mirror::ImageReplayer: 0x5636fba99c00 [3/790c038c-4353-4ff1-ae0b-9c9bb1e1adaa] start: already running 2021-02-20T22:04:57.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.941+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffce8000 [3/7fb27511-6f76-444f-b338-191b2c7caa61] start: already running 2021-02-20T22:04:57.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.945+0000 7f7f66b46700 -1 rbd::mirror::ImageReplayer: 0x5636ffce8280 [3/858c7e91-92b9-4e8b-8ae9-f8cb46e242bf] start: already running 2021-02-20T22:04:57.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.945+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffce8500 [3/87eed4f8-1d97-46e0-b439-8f6b85b99b40] start: already running 2021-02-20T22:04:57.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.945+0000 7f7f66b46700 -1 rbd::mirror::ImageReplayer: 0x5636ffce8780 [3/8c216315-5117-4846-b850-c4265b4697a3] start: already running 2021-02-20T22:04:57.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.945+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffce8a00 [3/91672d77-7c82-4d00-b945-306681ce602c] start: already running 2021-02-20T22:04:57.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.945+0000 7f7f66b46700 -1 rbd::mirror::ImageReplayer: 0x5636ffce8c80 [3/941323f6-0453-4944-ab79-664fde6ec580] start: already running 2021-02-20T22:04:57.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.945+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffce8f00 [3/945695ad-0f15-4fd7-a092-4e7df53e1d1b] start: already running 2021-02-20T22:04:57.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.945+0000 7f7f66b46700 -1 rbd::mirror::ImageReplayer: 0x5636ffce9180 [3/972a582c-01df-4309-8926-a24b908ec0fc] start: already running 2021-02-20T22:04:57.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.945+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffce9400 [3/9940675f-5679-4d59-9726-da9895d6bb7c] start: already running 2021-02-20T22:04:57.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.945+0000 7f7f66b46700 -1 rbd::mirror::ImageReplayer: 0x5636ffce9680 [3/9b42b7b2-0767-46b8-a6a6-e0b14c0863eb] start: already running 2021-02-20T22:04:57.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.945+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffce9900 [3/9bb53f74-14d3-4057-8f64-9bba63f946e2] start: already running 2021-02-20T22:04:57.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.949+0000 7f7f66b46700 -1 rbd::mirror::ImageReplayer: 0x5636ffce9b80 [3/a3cf9f38-0520-4671-b928-6d907b15b2fb] start: already running 2021-02-20T22:04:57.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.949+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffce9e00 [3/a3d3aab0-3900-425e-a825-80fa9ff092ea] start: already running 2021-02-20T22:04:57.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.949+0000 7f7f66b46700 -1 rbd::mirror::ImageReplayer: 0x5636ffcea080 [3/ab28f848-782d-4c9d-a26a-5bdb09404113] start: already running 2021-02-20T22:04:57.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.949+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffcea300 [3/b1b702c0-1f24-4c9b-b9bc-cc3a0752ca38] start: already running 2021-02-20T22:04:57.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.949+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffcea580 [3/b97c3e7d-cef7-4890-a315-ffa1f7e51804] start: already running 2021-02-20T22:04:57.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.949+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffcea800 [3/bbf1cffa-5e93-4c1c-937d-71c8c54806ff] start: already running 2021-02-20T22:04:57.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.949+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffceaa80 [3/cdea041d-e1fe-4928-9842-3ce9553bc980] start: already running 2021-02-20T22:04:57.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.949+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffcead00 [3/cdfd4b53-cfc9-488d-957e-07aa3d636acf] start: already running 2021-02-20T22:04:57.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.949+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffceaf80 [3/ce32d773-a477-4402-af84-407be17c35f5] start: already running 2021-02-20T22:04:57.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.949+0000 7f7f66b46700 -1 rbd::mirror::ImageReplayer: 0x5636ffceb200 [3/dd0c591d-efef-4cf8-9976-7731785fbcd4] start: already running 2021-02-20T22:04:57.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.949+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffceb480 [3/dd0f679e-1316-4b67-b5c5-1d6037eb2c90] start: already running 2021-02-20T22:04:57.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.949+0000 7f7f66b46700 -1 rbd::mirror::ImageReplayer: 0x5636ffceb700 [3/def81481-a32e-45b7-8067-6c32798fc165] start: already running 2021-02-20T22:04:57.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.949+0000 7f7f66b46700 -1 rbd::mirror::ImageReplayer: 0x5636ffceb980 [3/e5f43e4f-211a-45bf-bf8b-2e986b46e688] start: already running 2021-02-20T22:04:57.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.949+0000 7f7f66b46700 -1 rbd::mirror::ImageReplayer: 0x5636ffcebc00 [3/eafa574f-c2c8-4659-b559-be75ec7d56bb] start: already running 2021-02-20T22:04:57.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.949+0000 7f7f66b46700 -1 rbd::mirror::ImageReplayer: 0x5636ffd1a000 [3/ef164021-65e4-434b-a5d7-bcdd53df22fd] start: already running 2021-02-20T22:04:57.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.949+0000 7f7f66b46700 -1 rbd::mirror::ImageReplayer: 0x5636ffd1a280 [3/ff38d3a1-22f6-4a7c-a814-2d4505cf6570] start: already running 2021-02-20T22:04:57.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:57.949+0000 7f7f66b46700 -1 rbd::mirror::ImageReplayer: 0x5636ffd1a500 [3/ff64bb69-23c4-4b3a-9b65-a48c78854429] start: already running 2021-02-20T22:04:58.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:58.253+0000 7f7f66b46700 -1 rbd::mirror::ImageReplayer: 0x5636ffce9680 [3/9b42b7b2-0767-46b8-a6a6-e0b14c0863eb] register_admin_socket_hook: error registering admin socket commands 2021-02-20T22:04:58.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:58.253+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffce9680 [3/9b42b7b2-0767-46b8-a6a6-e0b14c0863eb] register_admin_socket_hook: error registering admin socket commands 2021-02-20T22:04:58.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:58.253+0000 7f7f66b46700 -1 rbd::mirror::ImageReplayer: 0x5636ffd1a000 [3/ef164021-65e4-434b-a5d7-bcdd53df22fd] register_admin_socket_hook: error registering admin socket commands 2021-02-20T22:04:58.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:04:58.253+0000 7f7f66345700 -1 rbd::mirror::ImageReplayer: 0x5636ffd1a000 [3/ef164021-65e4-434b-a5d7-bcdd53df22fd] register_admin_socket_hook: error registering admin socket commands 2021-02-20T22:05:00.575 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T22:05:00.576 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:05:00.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:05:00.576 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T22:05:00.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T22:05:00.577 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T22:05:00.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T22:05:00.580 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T22:05:52.703 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3362356938679901) > weight (0.25) 2021-02-20T22:05:52.710 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T22:05:52.711 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T22:05:52.712 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T22:05:52.713 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T22:05:52.708+0000 7fa4367fc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 74493) UID: 1000 2021-02-20T22:05:52.913 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:06:00.713 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T22:06:00.720 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:06:00.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:06:00.721 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T22:06:00.721 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T22:06:00.722 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T22:06:00.728 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T22:06:00.728 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T22:06:18.581 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4000198975549577) > weight (0.25) 2021-02-20T22:06:18.597 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T22:06:18.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T22:06:18.599 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T22:06:18.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T22:06:18.595+0000 7fba1230d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 74675) UID: 1000 2021-02-20T22:06:18.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:06:27.600 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T22:06:27.601 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:06:27.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:06:27.602 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T22:06:27.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T22:06:27.602 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T22:06:27.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T22:06:27.605 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T22:07:36.730 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3202606441700554) > weight (0.25) 2021-02-20T22:07:36.767 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6501096214822627) > weight (0.25) 2021-02-20T22:07:36.767 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T22:07:36.768 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T22:07:36.768 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T22:07:36.769 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T22:07:36.765+0000 7f6ef3320700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 74766) UID: 1000 2021-02-20T22:07:36.970 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:07:38.770 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T22:07:38.770 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:07:38.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:07:38.771 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T22:07:38.771 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T22:07:38.772 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T22:07:38.774 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T22:07:38.775 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T22:07:39.606 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8592682993541162) > weight (0.25) 2021-02-20T22:07:39.607 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9152033971573793) > weight (0.25) 2021-02-20T22:07:39.607 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.45900416305579894) > weight (0.25) 2021-02-20T22:07:39.607 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9878368339325553) > weight (0.25) 2021-02-20T22:07:39.608 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T22:08:46.775 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.566105549470833) > weight (0.25) 2021-02-20T22:08:46.783 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7777440872182209) > weight (0.25) 2021-02-20T22:08:46.783 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T22:08:46.784 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T22:08:46.784 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T22:08:46.785 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T22:08:46.779+0000 7fc0947c4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 75144) UID: 1000 2021-02-20T22:08:46.985 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:08:48.785 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T22:08:48.785 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:08:48.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:08:48.786 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T22:08:48.786 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T22:08:48.787 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T22:08:48.799 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T22:08:48.800 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T22:09:09.609 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T22:09:09.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T22:09:09.618 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T22:09:09.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T22:09:09.614+0000 7f617d755700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 74865) UID: 1000 2021-02-20T22:09:09.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:09:11.619 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T22:09:11.619 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:09:11.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:09:11.620 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T22:09:11.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T22:09:11.621 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T22:09:11.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T22:09:11.631 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T22:10:25.800 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T22:10:25.867 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T22:10:25.868 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T22:10:25.868 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T22:10:25.864+0000 7f96c07c8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 73223) UID: 1000 2021-02-20T22:10:26.069 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:10:33.869 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T22:10:33.878 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:10:33.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:10:33.879 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T22:10:33.879 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T22:10:33.880 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T22:10:33.883 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T22:10:33.883 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T22:10:48.471 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T22:10:48.464+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b5aa2f70 2021-02-20T22:10:48.471 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T22:10:48.464+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b66a6340 2021-02-20T22:10:48.472 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T22:10:48.468+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b66a7450 2021-02-20T22:10:48.472 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T22:10:48.468+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b8a8e410 2021-02-20T22:10:48.633 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9087316079745595) > weight (0.25) 2021-02-20T22:10:48.640 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8968279774170935) > weight (0.25) 2021-02-20T22:10:48.640 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T22:10:48.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T22:10:48.641 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T22:10:48.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T22:10:48.636+0000 7fae6b369700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 74217) UID: 1000 2021-02-20T22:10:48.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:10:53.642 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T22:10:53.642 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:10:53.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:10:53.643 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T22:10:53.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T22:10:53.644 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T22:10:53.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T22:10:53.647 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T22:11:40.884 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7995435736922794) > weight (0.25) 2021-02-20T22:11:41.195 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8702358342716944) > weight (0.25) 2021-02-20T22:11:41.195 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7671130598583745) > weight (0.25) 2021-02-20T22:11:41.195 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T22:11:41.196 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T22:11:41.196 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T22:11:41.197 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:11:41.190+0000 7fb800183700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 70342) UID: 1000 2021-02-20T22:11:41.398 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:11:42.197 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T22:11:42.198 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:11:42.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:11:42.198 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T22:11:42.199 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T22:11:42.199 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T22:11:42.213 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T22:11:42.213 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T22:12:04.239 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:12:04.233+0000 7fb8259e4700 -1 reset not still connected to 0x557848e73b50 2021-02-20T22:12:04.239 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:12:04.233+0000 7fb8259e4700 -1 reset not still connected to 0x557855c1d860 2021-02-20T22:12:04.246 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:12:04.237+0000 7fb8259e4700 -1 reset not still connected to 0x557855c1dba0 2021-02-20T22:12:04.248 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:12:04.237+0000 7fb8259e4700 -1 reset not still connected to 0x55785b75b040 2021-02-20T22:12:04.249 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:12:04.237+0000 7fb8259e4700 -1 reset not still connected to 0x55785d827380 2021-02-20T22:12:04.256 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:12:04.245+0000 7fb8259e4700 -1 reset not still connected to 0x55785f2ada80 2021-02-20T22:12:04.258 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:12:04.245+0000 7fb8259e4700 -1 reset not still connected to 0x557860c15330 2021-02-20T22:12:04.262 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:12:04.245+0000 7fb8259e4700 -1 reset not still connected to 0x557865e34270 2021-02-20T22:12:04.262 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:12:04.245+0000 7fb8259e4700 -1 reset not still connected to 0x557865e36970 2021-02-20T22:12:04.263 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:12:04.245+0000 7fb8259e4700 -1 reset not still connected to 0x557865e36ff0 2021-02-20T22:12:04.263 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:12:04.245+0000 7fb8259e4700 -1 reset not still connected to 0x557866b067d0 2021-02-20T22:12:19.648 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T22:12:19.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T22:12:19.657 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T22:12:19.657 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T22:12:19.657 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:12:19.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T22:12:19.657+0000 7f5246e46700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 75326) UID: 1000 2021-02-20T22:12:19.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:12:25.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:12:25.761 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T22:12:25.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T22:12:25.761 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T22:12:25.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T22:12:25.764 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T22:13:23.946 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T22:13:23.936+0000 7f4a1bb72700 -1 reset not still connected to 0x55fca629d520 2021-02-20T22:13:28.214 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T22:13:28.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T22:13:28.226 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T22:13:28.226 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T22:13:28.220+0000 7f625a90f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 75419) UID: 1000 2021-02-20T22:13:28.426 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:13:29.227 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T22:13:29.227 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:13:29.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:13:29.228 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T22:13:29.228 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T22:13:29.229 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T22:13:29.231 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T22:13:29.231 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T22:13:37.766 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8208547472341733) > weight (0.25) 2021-02-20T22:13:37.771 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9939646078799262) > weight (0.25) 2021-02-20T22:13:37.771 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6701853607537922) > weight (0.25) 2021-02-20T22:13:37.771 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7880507462104801) > weight (0.25) 2021-02-20T22:13:37.772 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T22:14:57.233 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T22:14:57.243 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T22:14:57.244 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T22:14:57.246 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T22:14:57.241+0000 7ff63bb60700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 75785) UID: 1000 2021-02-20T22:14:57.445 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:15:02.772 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.518154497786538) > weight (0.25) 2021-02-20T22:15:02.779 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8249364252843872) > weight (0.25) 2021-02-20T22:15:02.780 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T22:15:02.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T22:15:02.783 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T22:15:02.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T22:15:02.785+0000 7fe9a8b83700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 75510) UID: 1000 2021-02-20T22:15:02.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:15:05.245 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T22:15:05.246 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:15:05.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:15:05.246 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T22:15:05.247 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T22:15:05.247 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T22:15:05.250 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T22:15:05.250 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T22:15:09.784 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T22:15:09.785 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:15:09.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:15:09.786 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T22:15:09.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T22:15:09.787 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T22:15:09.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T22:15:09.792 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T22:16:26.253 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T22:16:26.282 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T22:16:26.284 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T22:16:26.285 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T22:16:26.279+0000 7f9720782700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 75879) UID: 1000 2021-02-20T22:16:26.484 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:16:33.286 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T22:16:33.286 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:16:33.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:16:33.287 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T22:16:33.287 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T22:16:33.288 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T22:16:33.290 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T22:16:33.290 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T22:17:00.793 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T22:17:00.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T22:17:00.802 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T22:17:00.803 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T22:17:00.803 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:17:00.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T22:17:00.798+0000 7f624bf97700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 75694) UID: 1000 2021-02-20T22:17:01.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:17:06.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:17:06.913 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T22:17:06.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T22:17:06.914 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T22:17:06.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T22:17:06.918 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T22:17:36.294 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4590431198462587) > weight (0.25) 2021-02-20T22:17:36.303 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8403074335278962) > weight (0.25) 2021-02-20T22:17:36.303 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7797282784702791) > weight (0.25) 2021-02-20T22:17:36.304 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T22:17:36.304 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T22:17:36.306 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T22:17:36.306 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:17:36.297+0000 7fc5230fb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 75602) UID: 1000 2021-02-20T22:17:36.506 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:17:44.307 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T22:17:44.307 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:17:44.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:17:44.308 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T22:17:44.308 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T22:17:44.308 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T22:17:44.311 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T22:17:44.311 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T22:18:12.919 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9851955135557533) > weight (0.25) 2021-02-20T22:18:12.927 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T22:18:12.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T22:18:12.929 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T22:18:12.929 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T22:18:12.929 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:18:12.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T22:18:12.924+0000 7ffbb479e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 75052) UID: 1000 2021-02-20T22:18:13.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:18:19.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:18:19.031 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T22:18:19.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T22:18:19.032 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T22:18:19.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T22:18:19.035 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T22:18:45.312 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8409483458250243) > weight (0.25) 2021-02-20T22:18:45.325 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.27909455017189533) > weight (0.25) 2021-02-20T22:18:45.325 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3098593624078815) > weight (0.25) 2021-02-20T22:18:45.325 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T22:18:45.326 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T22:18:45.327 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T22:18:45.327 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:18:45.323+0000 7f0aa7bb8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 76247) UID: 1000 2021-02-20T22:18:45.528 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:18:50.328 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T22:18:50.328 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:18:50.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:18:50.329 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T22:18:50.329 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T22:18:50.329 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T22:18:50.332 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T22:18:50.332 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T22:20:16.036 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6905987481399868) > weight (0.25) 2021-02-20T22:20:16.075 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T22:20:16.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T22:20:16.076 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T22:20:16.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T22:20:16.073+0000 7f38ee6ac700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 76339) UID: 1000 2021-02-20T22:20:16.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:20:18.078 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T22:20:18.084 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:20:18.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:20:18.085 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T22:20:18.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T22:20:18.086 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T22:20:18.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T22:20:18.090 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T22:20:39.334 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5301960362196687) > weight (0.25) 2021-02-20T22:20:39.356 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9305077056685679) > weight (0.25) 2021-02-20T22:20:39.356 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3988970449358539) > weight (0.25) 2021-02-20T22:20:39.357 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T22:20:39.357 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T22:20:39.357 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T22:20:39.358 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:20:39.356+0000 7fd9ab14e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 76430) UID: 1000 2021-02-20T22:20:39.559 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:20:41.358 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T22:20:41.359 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:20:41.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:20:41.360 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T22:20:41.361 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T22:20:41.361 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T22:20:41.365 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T22:20:41.366 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T22:21:49.091 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T22:21:49.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T22:21:49.102 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T22:21:49.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T22:21:49.098+0000 7f861da03700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 76156) UID: 1000 2021-02-20T22:21:49.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:21:54.103 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T22:21:55.532 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:21:55.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:21:55.533 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T22:21:55.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T22:21:55.533 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T22:21:55.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T22:21:55.539 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T22:22:33.367 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.774201822146994) > weight (0.25) 2021-02-20T22:22:33.376 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.46772287338338225) > weight (0.25) 2021-02-20T22:22:33.376 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T22:22:33.377 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T22:22:33.377 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T22:22:33.378 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T22:22:33.373+0000 7fe262a18700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 75235) UID: 1000 2021-02-20T22:22:33.579 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:22:35.378 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T22:22:35.378 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:22:35.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:22:35.379 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T22:22:35.380 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T22:22:35.380 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T22:22:35.383 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T22:22:35.384 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T22:23:09.540 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T22:23:09.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T22:23:09.563 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T22:23:09.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T22:23:09.560+0000 7fe445eb0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 76706) UID: 1000 2021-02-20T22:23:09.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:23:15.564 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T22:23:15.571 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:23:15.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:23:15.572 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T22:23:15.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T22:23:15.573 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T22:23:15.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T22:23:15.576 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T22:24:25.384 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T22:24:25.424 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T22:24:25.425 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T22:24:25.425 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T22:24:25.422+0000 7fcf3a81d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 76062) UID: 1000 2021-02-20T22:24:25.626 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:24:26.425 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T22:24:26.426 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:24:26.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:24:26.427 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T22:24:26.427 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T22:24:26.427 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T22:24:26.431 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T22:24:26.431 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T22:25:00.984 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T22:25:00.977+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a600dfb0 2021-02-20T22:25:00.990 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T22:25:00.981+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6cfb67e90 2021-02-20T22:25:00.990 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T22:25:00.981+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6e58ab040 2021-02-20T22:25:14.577 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8138663841598892) > weight (0.25) 2021-02-20T22:25:14.585 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3288982882539093) > weight (0.25) 2021-02-20T22:25:14.586 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T22:25:14.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T22:25:14.587 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T22:25:14.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T22:25:14.585+0000 7f68df476700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 75970) UID: 1000 2021-02-20T22:25:14.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:25:18.587 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T22:25:18.592 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:25:18.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:25:18.593 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T22:25:18.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T22:25:18.594 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T22:25:18.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T22:25:18.597 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T22:25:47.432 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9424756030094599) > weight (0.25) 2021-02-20T22:25:47.440 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3610479243472512) > weight (0.25) 2021-02-20T22:25:47.441 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6482439714229775) > weight (0.25) 2021-02-20T22:25:47.441 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6681122238563282) > weight (0.25) 2021-02-20T22:25:47.441 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T22:26:18.598 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5442167190644831) > weight (0.25) 2021-02-20T22:26:18.608 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.41687670368199725) > weight (0.25) 2021-02-20T22:26:18.608 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6309408485314121) > weight (0.25) 2021-02-20T22:26:18.609 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T22:26:18.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T22:26:18.611 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T22:26:18.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:26:18.607+0000 7f7f67347700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 72478) UID: 1000 2021-02-20T22:26:19.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:26:21.612 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T22:26:21.612 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:26:21.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:26:21.613 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T22:26:21.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T22:26:21.614 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T22:26:21.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T22:26:21.622 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T22:26:23.272 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:26:23.269+0000 7fb8259e4700 -1 reset not still connected to 0x557848e73190 2021-02-20T22:26:23.280 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:26:23.269+0000 7fb8259e4700 -1 reset not still connected to 0x557854d18150 2021-02-20T22:26:23.281 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:26:23.269+0000 7fb8259e4700 -1 reset not still connected to 0x557855c1e080 2021-02-20T22:26:23.281 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:26:23.269+0000 7fb8259e4700 -1 reset not still connected to 0x557858248a90 2021-02-20T22:26:23.281 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:26:23.269+0000 7fb8259e4700 -1 reset not still connected to 0x55785b75a000 2021-02-20T22:26:23.282 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:26:23.273+0000 7fb8259e4700 -1 reset not still connected to 0x55785c64e8f0 2021-02-20T22:26:23.282 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:26:23.273+0000 7fb8259e4700 -1 reset not still connected to 0x55785f2ab520 2021-02-20T22:26:23.282 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:26:23.273+0000 7fb8259e4700 -1 reset not still connected to 0x557863950c30 2021-02-20T22:26:23.282 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:26:23.273+0000 7fb8259e4700 -1 reset not still connected to 0x557863952be0 2021-02-20T22:27:20.442 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.37057662483114995) > weight (0.25) 2021-02-20T22:27:20.452 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7322770293518486) > weight (0.25) 2021-02-20T22:27:20.452 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5076081940124088) > weight (0.25) 2021-02-20T22:27:20.452 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5050187324904264) > weight (0.25) 2021-02-20T22:27:20.452 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T22:27:31.032 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:27:31.023+0000 7fb8259e4700 -1 reset not still connected to 0x557855c1c000 2021-02-20T22:27:31.041 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:27:31.027+0000 7fb8259e4700 -1 reset not still connected to 0x55785c64fd40 2021-02-20T22:27:52.623 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9507973178949054) > weight (0.25) 2021-02-20T22:27:52.631 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.26558700189408213) > weight (0.25) 2021-02-20T22:27:52.631 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T22:27:52.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T22:27:52.633 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T22:27:52.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T22:27:52.629+0000 7fbca32d6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 77074) UID: 1000 2021-02-20T22:27:52.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:27:55.634 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T22:27:55.635 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:27:55.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:27:55.635 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T22:27:55.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T22:27:55.636 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T22:27:55.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T22:27:55.640 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T22:29:08.454 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6443578969131026) > weight (0.25) 2021-02-20T22:29:08.463 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T22:29:08.464 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T22:29:08.465 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T22:29:08.466 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T22:29:08.459+0000 7f19e9358700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 74961) UID: 1000 2021-02-20T22:29:08.866 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:29:17.467 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T22:29:17.475 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:29:17.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:29:17.476 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T22:29:17.476 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T22:29:17.477 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T22:29:17.483 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T22:29:17.484 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T22:29:25.641 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3871343281090637) > weight (0.25) 2021-02-20T22:29:25.642 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T22:29:25.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T22:29:25.643 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T22:29:25.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T22:29:25.638+0000 7f46d90e0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 76522) UID: 1000 2021-02-20T22:29:26.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:29:26.643 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T22:29:26.644 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:29:26.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:29:26.644 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T22:29:26.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T22:29:26.645 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T22:29:26.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T22:29:26.648 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T22:30:35.671 INFO:tasks.ceph.cluster1.osd.1.smithi170.stderr:2021-02-20T22:30:35.663+0000 7fb102cd5700 -1 reset not still connected to 0x559d533cae50 2021-02-20T22:30:44.485 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.957943325437107) > weight (0.25) 2021-02-20T22:30:44.493 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.259754157660966) > weight (0.25) 2021-02-20T22:30:44.493 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5326640516668104) > weight (0.25) 2021-02-20T22:30:44.493 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5158306133290117) > weight (0.25) 2021-02-20T22:30:44.493 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T22:30:51.649 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.256531847752534) > weight (0.25) 2021-02-20T22:30:51.656 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.41276519643816734) > weight (0.25) 2021-02-20T22:30:51.656 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7398618312286913) > weight (0.25) 2021-02-20T22:30:51.657 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3317078396786881) > weight (0.25) 2021-02-20T22:30:51.657 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T22:31:59.495 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3634170886733241) > weight (0.25) 2021-02-20T22:31:59.503 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8991233069861987) > weight (0.25) 2021-02-20T22:31:59.504 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T22:31:59.504 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T22:31:59.505 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T22:31:59.505 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T22:31:59.505 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:31:59.506 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T22:31:59.502+0000 7f818a1e5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 76797) UID: 1000 2021-02-20T22:31:59.658 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6099459220041596) > weight (0.25) 2021-02-20T22:31:59.659 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6875595166957205) > weight (0.25) 2021-02-20T22:31:59.659 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.25746041926117214) > weight (0.25) 2021-02-20T22:31:59.659 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4794840576816267) > weight (0.25) 2021-02-20T22:31:59.659 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T22:31:59.706 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:32:05.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:32:05.609 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T22:32:05.609 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T22:32:05.610 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T22:32:05.612 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T22:32:05.613 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T22:33:04.660 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7204479121425387) > weight (0.25) 2021-02-20T22:33:04.669 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8094690516989315) > weight (0.25) 2021-02-20T22:33:04.669 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.95093533720372) > weight (0.25) 2021-02-20T22:33:04.670 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.41470333866326203) > weight (0.25) 2021-02-20T22:33:04.670 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T22:33:35.614 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8189341879383465) > weight (0.25) 2021-02-20T22:33:35.624 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.580992173750888) > weight (0.25) 2021-02-20T22:33:35.624 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T22:33:35.625 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T22:33:35.626 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T22:33:35.628 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T22:33:35.624+0000 7f3e0945d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 77550) UID: 1000 2021-02-20T22:33:35.829 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:33:36.627 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T22:33:36.628 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:33:36.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:33:36.629 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T22:33:36.629 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T22:33:36.629 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T22:33:36.633 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T22:33:36.633 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T22:34:07.670 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T22:34:07.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T22:34:07.679 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T22:34:07.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T22:34:07.679+0000 7f2423160700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 76889) UID: 1000 2021-02-20T22:34:07.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:34:13.680 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T22:34:13.687 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:34:13.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:34:13.688 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T22:34:13.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T22:34:13.688 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T22:34:13.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T22:34:13.720 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T22:34:53.635 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.30651663598714607) > weight (0.25) 2021-02-20T22:34:53.693 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3170573174028676) > weight (0.25) 2021-02-20T22:34:53.693 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T22:34:53.694 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T22:34:53.694 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T22:34:53.695 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T22:34:53.694+0000 7fb3dc1b2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 77641) UID: 1000 2021-02-20T22:34:53.896 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:34:56.696 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T22:34:56.696 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:34:56.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:34:56.697 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T22:34:56.697 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T22:34:56.698 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T22:34:56.700 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T22:34:56.700 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T22:34:56.857 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:34:56.853+0000 7fb8259e4700 -1 reset not still connected to 0x5578576bb380 2021-02-20T22:34:56.858 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:34:56.853+0000 7fb8259e4700 -1 reset not still connected to 0x5578582491e0 2021-02-20T22:35:25.833 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:35:25.828+0000 7fb8259e4700 -1 reset not still connected to 0x557854d18cb0 2021-02-20T22:35:25.836 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:35:25.828+0000 7fb8259e4700 -1 reset not still connected to 0x557855c1de10 2021-02-20T22:35:25.837 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:35:25.828+0000 7fb8259e4700 -1 reset not still connected to 0x5578576ba270 2021-02-20T22:35:25.837 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:35:25.828+0000 7fb8259e4700 -1 reset not still connected to 0x5578576ba410 2021-02-20T22:35:25.838 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:35:25.828+0000 7fb8259e4700 -1 reset not still connected to 0x5578576ba4e0 2021-02-20T22:35:25.838 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:35:25.828+0000 7fb8259e4700 -1 reset not still connected to 0x5578576bca40 2021-02-20T22:35:25.862 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:35:25.828+0000 7fb8259e4700 -1 reset not still connected to 0x5578576bd8e0 2021-02-20T22:35:25.864 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:35:25.828+0000 7fb8259e4700 -1 reset not still connected to 0x557858248a90 2021-02-20T22:35:25.864 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:35:25.828+0000 7fb8259e4700 -1 reset not still connected to 0x55785b75c3c0 2021-02-20T22:35:25.866 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:35:25.828+0000 7fb8259e4700 -1 reset not still connected to 0x55785d826f70 2021-02-20T22:35:25.873 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:35:25.828+0000 7fb8259e4700 -1 reset not still connected to 0x5578639508f0 2021-02-20T22:35:25.874 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:35:25.828+0000 7fb8259e4700 -1 reset not still connected to 0x557865e34ea0 2021-02-20T22:35:25.875 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T22:35:25.828+0000 7fb8259e4700 -1 reset not still connected to 0x557866b06970 2021-02-20T22:36:07.701 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.29418138011621453) > weight (0.25) 2021-02-20T22:36:07.753 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4317965910295398) > weight (0.25) 2021-02-20T22:36:07.753 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2609102311315491) > weight (0.25) 2021-02-20T22:36:07.754 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T22:36:07.754 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T22:36:07.756 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T22:36:07.756 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:36:07.752+0000 7f42e1a96700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 76613) UID: 1000 2021-02-20T22:36:07.957 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:36:09.722 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9838332153112441) > weight (0.25) 2021-02-20T22:36:09.728 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4352075832539186) > weight (0.25) 2021-02-20T22:36:09.729 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6423235346688854) > weight (0.25) 2021-02-20T22:36:09.729 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6481648783437499) > weight (0.25) 2021-02-20T22:36:09.729 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T22:36:16.757 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T22:36:16.763 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:36:16.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:36:16.764 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T22:36:16.764 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T22:36:16.766 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T22:36:16.769 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T22:36:16.769 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T22:37:23.730 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.404901861089373) > weight (0.25) 2021-02-20T22:37:23.766 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3058454030791802) > weight (0.25) 2021-02-20T22:37:23.767 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7010751733522033) > weight (0.25) 2021-02-20T22:37:23.767 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T22:37:23.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T22:37:23.768 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T22:37:23.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:37:23.762+0000 7fe5e73ac700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 77165) UID: 1000 2021-02-20T22:37:23.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:37:30.769 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T22:37:30.769 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:37:30.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:37:30.770 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T22:37:30.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T22:37:30.771 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T22:37:30.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T22:37:30.774 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T22:37:47.770 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8259119199893056) > weight (0.25) 2021-02-20T22:37:47.794 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4573458619653631) > weight (0.25) 2021-02-20T22:37:47.794 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9403113436827849) > weight (0.25) 2021-02-20T22:37:47.795 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T22:37:47.795 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T22:37:47.796 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T22:37:47.807 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:37:47.797+0000 7f01428f7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 77918) UID: 1000 2021-02-20T22:37:48.002 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:37:48.797 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T22:37:48.798 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:37:48.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:37:48.799 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T22:37:48.799 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T22:37:48.799 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T22:37:48.802 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T22:37:48.802 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T22:38:55.803 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4567040767377948) > weight (0.25) 2021-02-20T22:38:55.844 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6851495943989503) > weight (0.25) 2021-02-20T22:38:55.844 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.35427151355711806) > weight (0.25) 2021-02-20T22:38:55.845 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9472615890408576) > weight (0.25) 2021-02-20T22:38:55.845 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T22:39:20.775 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5993124356410656) > weight (0.25) 2021-02-20T22:39:20.787 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7159097992558037) > weight (0.25) 2021-02-20T22:39:20.787 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5261248433884738) > weight (0.25) 2021-02-20T22:39:20.787 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4147747916072485) > weight (0.25) 2021-02-20T22:39:20.787 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T22:40:00.846 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.42870230060016556) > weight (0.25) 2021-02-20T22:40:00.854 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T22:40:00.855 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T22:40:00.856 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T22:40:00.857 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T22:40:00.850+0000 7f8d08c3c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 77358) UID: 1000 2021-02-20T22:40:01.057 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:40:01.857 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T22:40:01.858 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:40:01.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:40:01.859 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T22:40:01.859 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T22:40:01.860 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T22:40:01.863 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T22:40:01.863 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T22:40:51.789 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7450779876345672) > weight (0.25) 2021-02-20T22:40:52.084 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T22:40:52.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T22:40:52.085 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T22:40:52.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T22:40:52.084+0000 7ff62d6f0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 77449) UID: 1000 2021-02-20T22:40:52.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:40:56.086 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T22:40:56.087 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:40:56.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:40:56.087 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T22:40:56.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T22:40:56.088 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T22:40:56.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T22:40:56.091 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T22:41:36.865 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.38804329461162146) > weight (0.25) 2021-02-20T22:41:36.877 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.372484648567025) > weight (0.25) 2021-02-20T22:41:36.878 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9701579266574806) > weight (0.25) 2021-02-20T22:41:36.878 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3145315629524682) > weight (0.25) 2021-02-20T22:41:36.879 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T22:42:14.125 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T22:42:14.118+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ca07f450 2021-02-20T22:42:38.880 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9303874272770198) > weight (0.25) 2021-02-20T22:42:38.887 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3930740592481734) > weight (0.25) 2021-02-20T22:42:38.887 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7544038700355622) > weight (0.25) 2021-02-20T22:42:38.888 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.30593426934724255) > weight (0.25) 2021-02-20T22:42:38.888 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T22:42:50.092 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5550073625408551) > weight (0.25) 2021-02-20T22:42:50.100 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9983166920858421) > weight (0.25) 2021-02-20T22:42:50.100 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.586896640683472) > weight (0.25) 2021-02-20T22:42:50.100 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.45425580906266383) > weight (0.25) 2021-02-20T22:42:50.100 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T22:43:43.889 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7826727963259119) > weight (0.25) 2021-02-20T22:43:43.913 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4280433022610498) > weight (0.25) 2021-02-20T22:43:43.914 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8959054353995388) > weight (0.25) 2021-02-20T22:43:43.914 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8789783156513552) > weight (0.25) 2021-02-20T22:43:43.914 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T22:44:48.101 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T22:44:48.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T22:44:48.115 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T22:44:48.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T22:44:48.118+0000 7f19b510a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 77733) UID: 1000 2021-02-20T22:44:48.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:44:50.116 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T22:44:50.117 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:44:50.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:44:50.118 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T22:44:50.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T22:44:50.123 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T22:44:50.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T22:44:50.126 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T22:45:01.915 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6926245900484641) > weight (0.25) 2021-02-20T22:45:01.924 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2558866744069769) > weight (0.25) 2021-02-20T22:45:01.924 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.30520165293603607) > weight (0.25) 2021-02-20T22:45:01.924 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9800430169713081) > weight (0.25) 2021-02-20T22:45:01.925 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T22:46:01.926 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T22:46:02.222 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T22:46:02.223 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T22:46:02.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T22:46:02.220+0000 7f9a58230700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 76980) UID: 1000 2021-02-20T22:46:02.624 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:46:05.223 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T22:46:05.224 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:46:05.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:46:05.225 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T22:46:05.225 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T22:46:05.225 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T22:46:05.243 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T22:46:05.244 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T22:46:41.128 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T22:46:41.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T22:46:41.135 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T22:46:41.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T22:46:41.131+0000 7f59da822700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 78378) UID: 1000 2021-02-20T22:46:41.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:46:44.136 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T22:46:44.140 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:46:44.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:46:44.141 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T22:46:44.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T22:46:44.142 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T22:46:44.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T22:46:44.146 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T22:47:35.244 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T22:47:35.257 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T22:47:35.257 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T22:47:35.261 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T22:47:35.258+0000 7fb720bb6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 78474) UID: 1000 2021-02-20T22:47:35.461 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:47:36.259 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T22:47:36.259 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:47:36.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:47:36.260 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T22:47:36.260 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T22:47:36.261 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T22:47:36.266 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T22:47:36.266 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T22:48:41.147 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.28876253748146163) > weight (0.25) 2021-02-20T22:48:41.179 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.667820537612971) > weight (0.25) 2021-02-20T22:48:41.180 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7971845914260748) > weight (0.25) 2021-02-20T22:48:41.180 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6939307433364982) > weight (0.25) 2021-02-20T22:48:41.180 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T22:49:19.267 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9717641550136972) > weight (0.25) 2021-02-20T22:49:19.280 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9563900764655554) > weight (0.25) 2021-02-20T22:49:19.281 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T22:49:19.282 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T22:49:19.282 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T22:49:19.284 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T22:49:19.279+0000 7f33bdaa3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 77824) UID: 1000 2021-02-20T22:49:19.483 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:49:21.283 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T22:49:21.283 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:49:21.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:49:21.284 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T22:49:21.284 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T22:49:21.285 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T22:49:21.288 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T22:49:21.288 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T22:49:23.508 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.503+0000 7f652d009700 -1 rbd::mirror::ImageReplayer: 0x557bbe6ffb80 [3/172baefb-5ce5-45e0-997e-249560de6fbd] start: already running 2021-02-20T22:49:23.509 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.503+0000 7f652d009700 -1 rbd::mirror::ImageReplayer: 0x557bbe700d00 [3/49a0b5eb-7d69-4824-806b-1ece12354eb9] start: already running 2021-02-20T22:49:23.512 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.503+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bbe6ffb80 [3/172baefb-5ce5-45e0-997e-249560de6fbd] start: already running 2021-02-20T22:49:23.513 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.503+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bbe6ffe00 [3/1a170226-518b-4a75-85a8-6fbdc9b5d9f0] start: already running 2021-02-20T22:49:23.513 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.503+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bbe700080 [3/205dce0d-bc7c-41b6-8ed9-87e6a63d1a70] start: already running 2021-02-20T22:49:23.513 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.503+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bbe700300 [3/2464eeeb-6954-426d-a8e5-129943cea55c] start: already running 2021-02-20T22:49:23.514 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bbe700580 [3/3ab5666f-763c-442d-bc17-ea037dcc3a42] start: already running 2021-02-20T22:49:23.514 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bbe700800 [3/3c6940ee-a642-422e-aafb-e3920bba87cf] start: already running 2021-02-20T22:49:23.515 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bbe700a80 [3/3d38caf8-6420-4f62-95dc-628caa5144da] start: already running 2021-02-20T22:49:23.515 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bbe700d00 [3/49a0b5eb-7d69-4824-806b-1ece12354eb9] start: already running 2021-02-20T22:49:23.515 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bbe700f80 [3/4c862af6-d1da-4215-b6b8-461f3883b104] start: already running 2021-02-20T22:49:23.516 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bbe701200 [3/4cba354e-99d2-4859-a4d0-baeec2203656] start: already running 2021-02-20T22:49:23.516 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bbe701480 [3/567ace7d-44e4-4d42-b47a-38bd35ec0e11] start: already running 2021-02-20T22:49:23.516 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bbe701700 [3/5786ec48-0da5-470d-b6a6-ddd71a666657] start: already running 2021-02-20T22:49:23.516 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bbe701980 [3/5c036645-b242-4fa8-bde0-d5b28dda3fd3] start: already running 2021-02-20T22:49:23.517 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bbe701c00 [3/5f3204b5-90c6-4244-a940-e749860457d5] start: already running 2021-02-20T22:49:23.517 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bc277a000 [3/675a1b7b-4e17-4b1e-9e92-eaf406f1c33f] start: already running 2021-02-20T22:49:23.517 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bc277a280 [3/681dbea5-7557-4dba-a92d-01da85613e31] start: already running 2021-02-20T22:49:23.517 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bc277a500 [3/753c14e7-97d4-4109-8814-2c7f1abbbee8] start: already running 2021-02-20T22:49:23.518 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bc277a780 [3/7658e9e0-5116-4c47-bad5-4e7894e27da7] start: already running 2021-02-20T22:49:23.518 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bc277aa00 [3/790c038c-4353-4ff1-ae0b-9c9bb1e1adaa] start: already running 2021-02-20T22:49:23.518 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bc277ac80 [3/7fb27511-6f76-444f-b338-191b2c7caa61] start: already running 2021-02-20T22:49:23.519 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bc277af00 [3/858c7e91-92b9-4e8b-8ae9-f8cb46e242bf] start: already running 2021-02-20T22:49:23.519 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bc277b180 [3/87eed4f8-1d97-46e0-b439-8f6b85b99b40] start: already running 2021-02-20T22:49:23.519 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bc277b400 [3/8c216315-5117-4846-b850-c4265b4697a3] start: already running 2021-02-20T22:49:23.519 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bc277b680 [3/91672d77-7c82-4d00-b945-306681ce602c] start: already running 2021-02-20T22:49:23.520 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bc277b900 [3/941323f6-0453-4944-ab79-664fde6ec580] start: already running 2021-02-20T22:49:23.520 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bc277bb80 [3/945695ad-0f15-4fd7-a092-4e7df53e1d1b] start: already running 2021-02-20T22:49:23.520 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bc277be00 [3/972a582c-01df-4309-8926-a24b908ec0fc] start: already running 2021-02-20T22:49:23.520 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bc277c080 [3/9940675f-5679-4d59-9726-da9895d6bb7c] start: already running 2021-02-20T22:49:23.521 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bc277c300 [3/9b42b7b2-0767-46b8-a6a6-e0b14c0863eb] start: already running 2021-02-20T22:49:23.521 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bc277c580 [3/9bb53f74-14d3-4057-8f64-9bba63f946e2] start: already running 2021-02-20T22:49:23.521 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bc277c800 [3/a3cf9f38-0520-4671-b928-6d907b15b2fb] start: already running 2021-02-20T22:49:23.521 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T22:49:23.507+0000 7f652d80a700 -1 rbd::mirror::ImageReplayer: 0x557bc277ca80 [3/a3d3aab0-3900-425e-a825-80fa9ff092ea] start: already running 2021-02-20T22:50:03.181 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6799686956231159) > weight (0.25) 2021-02-20T22:50:03.191 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5029860274507386) > weight (0.25) 2021-02-20T22:50:03.191 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6212667255218025) > weight (0.25) 2021-02-20T22:50:03.192 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9287189968259694) > weight (0.25) 2021-02-20T22:50:03.192 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T22:50:45.289 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5607802579950603) > weight (0.25) 2021-02-20T22:50:45.323 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T22:50:45.323 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T22:50:45.324 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T22:50:45.325 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T22:50:45.325+0000 7f319e65d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 78193) UID: 1000 2021-02-20T22:50:45.525 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:50:47.325 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T22:50:47.325 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:50:47.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:50:47.326 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T22:50:47.327 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T22:50:47.328 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T22:50:47.331 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T22:50:47.332 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T22:51:49.333 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8966659551403485) > weight (0.25) 2021-02-20T22:51:49.345 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T22:51:49.346 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T22:51:49.347 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T22:51:49.348 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T22:51:49.343+0000 7f39b5fbc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 78841) UID: 1000 2021-02-20T22:51:49.547 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:51:56.348 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T22:51:56.353 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:51:56.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:51:56.353 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T22:51:56.354 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T22:51:56.354 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T22:51:56.357 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T22:51:56.357 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T22:52:01.193 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8109511299520211) > weight (0.25) 2021-02-20T22:52:01.202 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.764766726702207) > weight (0.25) 2021-02-20T22:52:01.203 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6474070514534831) > weight (0.25) 2021-02-20T22:52:01.203 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T22:52:01.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T22:52:01.204 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T22:52:01.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T22:52:01.199+0000 7f3ce3435700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 78010) UID: 1000 2021-02-20T22:52:01.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:52:07.205 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T22:52:07.205 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:52:07.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:52:07.206 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T22:52:07.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T22:52:07.207 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T22:52:07.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T22:52:07.210 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T22:53:41.211 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8941428554162631) > weight (0.25) 2021-02-20T22:53:41.291 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.37904230593147004) > weight (0.25) 2021-02-20T22:53:41.291 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8769103154966095) > weight (0.25) 2021-02-20T22:53:41.292 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3746543927553381) > weight (0.25) 2021-02-20T22:53:41.292 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T22:53:51.358 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.39660530058181) > weight (0.25) 2021-02-20T22:53:51.367 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3844514244761036) > weight (0.25) 2021-02-20T22:53:51.367 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T22:53:51.368 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T22:53:51.368 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T22:53:51.370 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T22:53:51.368+0000 7f4dce2b2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 78748) UID: 1000 2021-02-20T22:53:51.570 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:53:57.369 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T22:53:57.369 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:53:57.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:53:57.370 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T22:53:57.370 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T22:53:57.371 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T22:53:57.378 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T22:53:57.379 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T22:55:13.380 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T22:55:13.404 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T22:55:13.406 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T22:55:13.406 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T22:55:13.401+0000 7f34ba555700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 78657) UID: 1000 2021-02-20T22:55:13.605 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:55:15.406 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T22:55:15.406 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:55:15.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:55:15.407 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T22:55:15.407 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T22:55:15.408 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T22:55:15.410 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T22:55:15.411 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T22:55:21.294 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T22:55:21.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T22:55:21.303 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T22:55:21.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T22:55:21.297+0000 7f10ff8df700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 78565) UID: 1000 2021-02-20T22:55:21.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:55:22.304 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T22:55:22.305 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:55:22.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:55:22.305 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T22:55:22.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T22:55:22.306 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T22:55:22.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T22:55:22.319 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T22:56:25.320 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T22:56:25.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T22:56:25.329 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T22:56:25.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T22:56:25.328+0000 7f572821d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 79301) UID: 1000 2021-02-20T22:56:25.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:56:29.330 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T22:56:29.339 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:56:29.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:56:29.339 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T22:56:29.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T22:56:29.342 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T22:56:29.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T22:56:29.348 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T22:56:46.411 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5484665824260179) > weight (0.25) 2021-02-20T22:56:46.419 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8755516609839197) > weight (0.25) 2021-02-20T22:56:46.419 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T22:56:46.420 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T22:56:46.420 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T22:56:46.421 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T22:56:46.419+0000 7fec6fd8a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 79116) UID: 1000 2021-02-20T22:56:46.621 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:56:50.422 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T22:56:50.423 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:56:50.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:56:50.426 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T22:56:50.429 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T22:56:50.434 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T22:56:50.444 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T22:56:50.444 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T22:57:36.348 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8146445788224809) > weight (0.25) 2021-02-20T22:57:36.357 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3863910827546606) > weight (0.25) 2021-02-20T22:57:36.357 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9402641105262601) > weight (0.25) 2021-02-20T22:57:36.358 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.44226387369162956) > weight (0.25) 2021-02-20T22:57:36.358 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T22:57:55.445 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T22:57:55.453 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T22:57:55.454 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T22:57:55.455 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T22:57:55.449+0000 7fe3cf5ff700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 79210) UID: 1000 2021-02-20T22:57:55.655 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:58:04.455 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T22:58:04.463 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:58:04.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:58:04.465 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T22:58:04.465 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T22:58:04.465 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T22:58:04.468 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T22:58:04.469 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T22:58:26.840 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T22:58:26.836+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a600fc20 2021-02-20T22:58:26.841 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T22:58:26.840+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b66a6820 2021-02-20T22:58:26.842 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T22:58:26.840+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b9ff6a40 2021-02-20T22:58:26.842 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T22:58:26.840+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6bab4c4e0 2021-02-20T22:58:26.843 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T22:58:26.840+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6bab4d040 2021-02-20T22:58:26.847 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T22:58:26.844+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6bab4d790 2021-02-20T22:58:26.847 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T22:58:26.844+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6bab4e080 2021-02-20T22:58:26.848 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T22:58:26.844+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6ca07e9c0 2021-02-20T22:58:26.848 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T22:58:26.844+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6cfb649c0 2021-02-20T22:58:26.849 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T22:58:26.848+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6cfb668a0 2021-02-20T22:58:26.849 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T22:58:26.848+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6e58ab040 2021-02-20T22:59:01.359 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.959457747589479) > weight (0.25) 2021-02-20T22:59:01.367 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T22:59:01.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T22:59:01.369 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T22:59:01.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T22:59:01.367+0000 7f61a6787700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 78285) UID: 1000 2021-02-20T22:59:01.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:59:03.370 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T22:59:03.370 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:59:03.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:59:03.371 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T22:59:03.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T22:59:03.372 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T22:59:03.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T22:59:03.375 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T22:59:09.470 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T22:59:09.477 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T22:59:09.477 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T22:59:09.478 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T22:59:09.475+0000 7f3f6c925700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 79576) UID: 1000 2021-02-20T22:59:09.678 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T22:59:11.478 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T22:59:11.499 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T22:59:11.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T22:59:11.500 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T22:59:11.500 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T22:59:11.501 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T22:59:11.506 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T22:59:11.507 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T23:00:52.378 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.25329869163829244) > weight (0.25) 2021-02-20T23:00:52.409 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T23:00:52.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T23:00:52.411 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T23:00:52.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T23:00:52.408+0000 7f71425c8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 79667) UID: 1000 2021-02-20T23:00:52.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:00:57.412 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T23:00:57.420 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:00:57.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:00:57.421 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T23:00:57.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T23:00:57.422 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T23:00:57.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T23:00:57.425 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T23:01:01.508 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2781420373430007) > weight (0.25) 2021-02-20T23:01:01.515 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T23:01:01.516 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T23:01:01.517 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T23:01:01.518 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T23:01:01.516+0000 7f986083c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 78932) UID: 1000 2021-02-20T23:01:01.718 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:01:08.518 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T23:01:08.525 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:01:08.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:01:08.526 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T23:01:08.526 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T23:01:08.527 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T23:01:08.530 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T23:01:08.530 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T23:01:19.331 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T23:01:19.311+0000 7fb8259e4700 -1 reset not still connected to 0x557863b44820 2021-02-20T23:01:19.332 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T23:01:19.315+0000 7fb8259e4700 -1 reset not still connected to 0x557866b052b0 2021-02-20T23:01:59.427 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T23:01:59.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T23:01:59.436 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T23:01:59.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T23:01:59.435+0000 7f0fc9759700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 79393) UID: 1000 2021-02-20T23:01:59.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:02:00.437 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T23:02:00.438 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:02:00.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:02:00.439 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T23:02:00.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T23:02:00.439 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T23:02:00.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T23:02:00.443 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T23:03:02.444 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4213630240018803) > weight (0.25) 2021-02-20T23:03:02.485 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.46022776798032916) > weight (0.25) 2021-02-20T23:03:02.485 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4269980138178898) > weight (0.25) 2021-02-20T23:03:02.486 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T23:03:02.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T23:03:02.487 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T23:03:02.487 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T23:03:02.488 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:03:02.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T23:03:02.481+0000 7f0658dd7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 79024) UID: 1000 2021-02-20T23:03:02.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:03:03.532 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T23:03:03.538 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T23:03:03.540 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T23:03:03.540 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T23:03:03.533+0000 7fe2552b2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 79758) UID: 1000 2021-02-20T23:03:03.740 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:03:08.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:03:08.590 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T23:03:08.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T23:03:08.591 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T23:03:08.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T23:03:08.594 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T23:03:12.541 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T23:03:12.541 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:03:12.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:03:12.542 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T23:03:12.542 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T23:03:12.542 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T23:03:12.545 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T23:03:12.545 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T23:04:48.595 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.46554347435079746) > weight (0.25) 2021-02-20T23:04:48.648 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.48309760490317033) > weight (0.25) 2021-02-20T23:04:48.649 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2855709247294589) > weight (0.25) 2021-02-20T23:04:48.649 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T23:04:48.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T23:04:48.650 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T23:04:48.651 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T23:04:48.651 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:04:48.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T23:04:48.650+0000 7f84ab017700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 80128) UID: 1000 2021-02-20T23:04:48.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:04:54.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:04:54.779 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T23:04:54.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T23:04:54.780 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T23:04:54.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T23:04:54.785 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T23:05:03.559 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.37734650584717444) > weight (0.25) 2021-02-20T23:05:03.567 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8999302140200264) > weight (0.25) 2021-02-20T23:05:03.567 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8370144551227773) > weight (0.25) 2021-02-20T23:05:03.567 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.543945637398591) > weight (0.25) 2021-02-20T23:05:03.567 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T23:06:12.786 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T23:06:12.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T23:06:12.800 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T23:06:12.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T23:06:12.800+0000 7f4c2daf5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 80034) UID: 1000 2021-02-20T23:06:13.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:06:21.801 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T23:06:21.806 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:06:21.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:06:21.807 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T23:06:21.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T23:06:21.807 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T23:06:21.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T23:06:21.811 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T23:06:28.570 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T23:06:28.581 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T23:06:28.582 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T23:06:28.582 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T23:06:28.580+0000 7f672c6df700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 80219) UID: 1000 2021-02-20T23:06:28.783 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:06:29.582 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T23:06:29.583 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:06:29.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:06:29.584 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T23:06:29.584 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T23:06:29.584 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T23:06:29.587 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T23:06:29.587 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T23:07:21.812 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3925934856019404) > weight (0.25) 2021-02-20T23:07:21.820 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3085957962123359) > weight (0.25) 2021-02-20T23:07:21.820 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6686184386332193) > weight (0.25) 2021-02-20T23:07:21.821 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6471476564636867) > weight (0.25) 2021-02-20T23:07:21.821 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T23:08:19.588 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T23:08:19.597 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T23:08:19.598 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T23:08:19.599 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T23:08:19.597+0000 7f6953460700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 80495) UID: 1000 2021-02-20T23:08:19.799 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:08:27.599 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T23:08:27.613 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:08:27.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:08:27.614 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T23:08:27.614 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T23:08:27.614 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T23:08:27.617 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T23:08:27.618 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T23:09:05.822 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T23:09:05.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T23:09:05.867 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T23:09:05.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T23:09:05.867+0000 7ff265db8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 80404) UID: 1000 2021-02-20T23:09:06.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:09:09.868 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T23:09:09.869 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:09:09.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:09:09.870 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T23:09:09.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T23:09:09.870 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T23:09:09.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T23:09:09.873 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T23:09:35.619 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T23:09:35.628 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T23:09:35.630 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T23:09:35.631 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T23:09:35.627+0000 7f4698c64700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 80588) UID: 1000 2021-02-20T23:09:35.831 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:09:38.631 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T23:09:38.631 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:09:38.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:09:38.632 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T23:09:38.632 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T23:09:38.633 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T23:09:38.656 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T23:09:38.657 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T23:10:21.875 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.592157191582416) > weight (0.25) 2021-02-20T23:10:21.885 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T23:10:21.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T23:10:21.886 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T23:10:21.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T23:10:21.881+0000 7fad93a73700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 79851) UID: 1000 2021-02-20T23:10:22.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:10:28.887 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T23:10:28.894 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:10:28.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:10:28.895 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T23:10:28.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T23:10:28.896 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T23:10:28.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T23:10:28.901 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T23:11:22.658 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T23:11:22.696 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T23:11:22.697 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T23:11:22.697 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T23:11:22.696+0000 7f73bdca4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 80770) UID: 1000 2021-02-20T23:11:22.898 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:11:26.697 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T23:11:26.705 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:11:26.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:11:26.706 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T23:11:26.707 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T23:11:26.707 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T23:11:26.711 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T23:11:26.712 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T23:12:13.902 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4261746657754897) > weight (0.25) 2021-02-20T23:12:13.911 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.37068542728421394) > weight (0.25) 2021-02-20T23:12:13.911 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5241687958384876) > weight (0.25) 2021-02-20T23:12:13.912 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.2586969993271089) > weight (0.25) 2021-02-20T23:12:13.912 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T23:12:27.852 INFO:tasks.ceph.cluster1.osd.0.smithi170.stderr:2021-02-20T23:12:27.843+0000 7f184c55a700 -1 reset not still connected to 0x5655107f19b0 2021-02-20T23:12:27.853 INFO:tasks.ceph.cluster1.osd.0.smithi170.stderr:2021-02-20T23:12:27.843+0000 7f184c55a700 -1 reset not still connected to 0x5655107f1cf0 2021-02-20T23:12:56.713 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.26892869461933633) > weight (0.25) 2021-02-20T23:12:56.720 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.510414455024068) > weight (0.25) 2021-02-20T23:12:56.720 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.46124425673201486) > weight (0.25) 2021-02-20T23:12:56.720 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.25248731296440863) > weight (0.25) 2021-02-20T23:12:56.721 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T23:13:15.913 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5362279350671905) > weight (0.25) 2021-02-20T23:13:15.920 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7510229210042776) > weight (0.25) 2021-02-20T23:13:15.920 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7072744706568911) > weight (0.25) 2021-02-20T23:13:15.920 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8034988767381062) > weight (0.25) 2021-02-20T23:13:15.921 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T23:14:53.722 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7021634965441014) > weight (0.25) 2021-02-20T23:14:53.729 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4905892347512475) > weight (0.25) 2021-02-20T23:14:53.729 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6372210606679556) > weight (0.25) 2021-02-20T23:14:53.730 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6784655498089515) > weight (0.25) 2021-02-20T23:14:53.730 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T23:15:06.922 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7130081276775155) > weight (0.25) 2021-02-20T23:15:06.930 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T23:15:06.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T23:15:06.935 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T23:15:06.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T23:15:06.930+0000 7f8245768700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 80862) UID: 1000 2021-02-20T23:15:07.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:15:11.936 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T23:15:11.960 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:15:11.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:15:11.961 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T23:15:11.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T23:15:11.961 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T23:15:11.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T23:15:11.965 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T23:15:58.731 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T23:15:58.741 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T23:15:58.741 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T23:15:58.742 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T23:15:58.740+0000 7f6c18ae2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 80954) UID: 1000 2021-02-20T23:15:58.942 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:16:02.742 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T23:16:02.743 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:16:02.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:16:02.743 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T23:16:02.744 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T23:16:02.744 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T23:16:02.747 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T23:16:02.747 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T23:17:07.966 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8522360313096377) > weight (0.25) 2021-02-20T23:17:07.975 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7407169629794921) > weight (0.25) 2021-02-20T23:17:07.975 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.517497757269496) > weight (0.25) 2021-02-20T23:17:07.975 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T23:17:07.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T23:17:07.977 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T23:17:07.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T23:17:07.975+0000 7f0f424e2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 80310) UID: 1000 2021-02-20T23:17:08.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:17:11.977 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T23:17:11.982 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:17:11.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:17:11.983 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T23:17:11.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T23:17:11.984 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T23:17:11.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T23:17:11.987 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T23:17:39.748 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6582124796086462) > weight (0.25) 2021-02-20T23:17:39.758 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.901083495324799) > weight (0.25) 2021-02-20T23:17:39.758 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T23:17:39.759 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T23:17:39.760 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T23:17:39.761 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T23:17:39.758+0000 7f1828a05700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 79484) UID: 1000 2021-02-20T23:17:39.961 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:17:46.761 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T23:17:46.762 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:17:46.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:17:46.763 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T23:17:46.763 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T23:17:46.763 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T23:17:46.766 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T23:17:46.767 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T23:18:43.997 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.44941899921134754) > weight (0.25) 2021-02-20T23:18:44.005 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.612789110223039) > weight (0.25) 2021-02-20T23:18:44.006 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.43289897237565855) > weight (0.25) 2021-02-20T23:18:44.006 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8097555227962838) > weight (0.25) 2021-02-20T23:18:44.006 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T23:19:43.769 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3770271072477226) > weight (0.25) 2021-02-20T23:19:43.777 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T23:19:43.778 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T23:19:43.779 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T23:19:43.781 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T23:19:43.778+0000 7f426f0fa700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 79942) UID: 1000 2021-02-20T23:19:43.980 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:19:52.780 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T23:19:52.786 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:19:52.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:19:52.787 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T23:19:52.788 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T23:19:52.789 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T23:19:52.793 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T23:19:52.793 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T23:20:16.008 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3165804547763811) > weight (0.25) 2021-02-20T23:20:16.016 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3590694596345536) > weight (0.25) 2021-02-20T23:20:16.016 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T23:20:16.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T23:20:16.018 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T23:20:16.018 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T23:20:16.018 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:20:16.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T23:20:16.018+0000 7f2fce5fe700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 77266) UID: 1000 2021-02-20T23:20:16.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:20:22.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:20:22.122 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T23:20:22.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T23:20:22.123 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T23:20:22.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T23:20:22.130 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T23:21:08.794 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.47076290330590465) > weight (0.25) 2021-02-20T23:21:08.803 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6017542297784088) > weight (0.25) 2021-02-20T23:21:08.804 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5247936807681318) > weight (0.25) 2021-02-20T23:21:08.804 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T23:21:08.805 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T23:21:08.805 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T23:21:08.807 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T23:21:08.804+0000 7f652e00b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 78101) UID: 1000 2021-02-20T23:21:09.207 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:21:10.806 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T23:21:10.806 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:21:10.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:21:10.807 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T23:21:10.807 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T23:21:10.808 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T23:21:10.810 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T23:21:10.811 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T23:21:30.132 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7994546421421518) > weight (0.25) 2021-02-20T23:21:30.147 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3729790954876636) > weight (0.25) 2021-02-20T23:21:30.148 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.41850404943957364) > weight (0.25) 2021-02-20T23:21:30.148 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.93528997313486) > weight (0.25) 2021-02-20T23:21:30.148 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T23:22:47.150 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T23:22:47.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T23:22:47.159 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T23:22:47.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T23:22:47.158+0000 7f881421a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 80679) UID: 1000 2021-02-20T23:22:47.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:22:52.811 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3024496188588308) > weight (0.25) 2021-02-20T23:22:52.812 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.780129284960676) > weight (0.25) 2021-02-20T23:22:52.812 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.32986324952128276) > weight (0.25) 2021-02-20T23:22:52.812 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.25670507441591883) > weight (0.25) 2021-02-20T23:22:52.813 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T23:22:53.159 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T23:22:53.160 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:22:53.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:22:53.161 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T23:22:53.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T23:22:53.162 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T23:22:53.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T23:22:53.165 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T23:23:26.530 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T23:23:26.523+0000 7fb8259e4700 -1 reset not still connected to 0x55785d828560 2021-02-20T23:23:26.531 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T23:23:26.523+0000 7fb8259e4700 -1 reset not still connected to 0x55785d828cb0 2021-02-20T23:23:26.531 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T23:23:26.523+0000 7fb8259e4700 -1 reset not still connected to 0x557865e368a0 2021-02-20T23:23:55.165 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.55288080923821) > weight (0.25) 2021-02-20T23:23:55.166 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T23:23:55.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T23:23:55.167 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T23:23:55.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T23:23:55.168+0000 7fd8f2981700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 81050) UID: 1000 2021-02-20T23:23:55.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:24:04.168 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T23:24:04.168 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:24:04.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:24:04.169 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T23:24:04.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T23:24:04.170 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T23:24:04.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T23:24:04.173 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T23:24:26.813 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T23:24:26.814 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T23:24:26.815 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T23:24:26.816 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T23:24:26.811+0000 7fcd6ff64700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 81142) UID: 1000 2021-02-20T23:24:27.016 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:24:34.816 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T23:24:34.816 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:24:34.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:24:34.817 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T23:24:34.818 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T23:24:34.818 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T23:24:34.821 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T23:24:34.822 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T23:25:40.174 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5393729122402366) > weight (0.25) 2021-02-20T23:25:40.175 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6034183710512394) > weight (0.25) 2021-02-20T23:25:40.175 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8309070173550804) > weight (0.25) 2021-02-20T23:25:40.175 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3982784993109487) > weight (0.25) 2021-02-20T23:25:40.176 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T23:26:19.823 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4277141411291999) > weight (0.25) 2021-02-20T23:26:19.830 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5633704774654348) > weight (0.25) 2021-02-20T23:26:19.831 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.989672220196926) > weight (0.25) 2021-02-20T23:26:19.831 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6716225990690169) > weight (0.25) 2021-02-20T23:26:19.831 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T23:26:51.177 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T23:26:51.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T23:26:51.179 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T23:26:51.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T23:26:51.179+0000 7f7228114700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 81719) UID: 1000 2021-02-20T23:26:51.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:26:59.180 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T23:26:59.180 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:26:59.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:26:59.181 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T23:26:59.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T23:26:59.182 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T23:26:59.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T23:26:59.185 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T23:27:29.833 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.29030897546315737) > weight (0.25) 2021-02-20T23:27:29.833 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4358256727694123) > weight (0.25) 2021-02-20T23:27:29.833 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T23:27:29.834 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T23:27:29.834 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T23:27:29.835 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T23:27:29.834+0000 7ff712a02700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 81329) UID: 1000 2021-02-20T23:27:30.036 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:27:32.835 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T23:27:32.836 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:27:32.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:27:32.837 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T23:27:32.837 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T23:27:32.837 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T23:27:32.840 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T23:27:32.840 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T23:27:49.960 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T23:27:49.954+0000 7f03ccd38700 -1 reset not still connected to 0x556b8baaa220 2021-02-20T23:27:49.961 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-20T23:27:49.954+0000 7f03ccd38700 -1 reset not still connected to 0x556b93878220 2021-02-20T23:27:53.093 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-20T23:27:53.089+0000 7fb8259e4700 -1 reset not still connected to 0x557863952970 2021-02-20T23:28:41.842 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T23:28:41.843 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T23:28:41.843 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T23:28:41.844 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T23:28:41.844+0000 7fabc17bb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 81902) UID: 1000 2021-02-20T23:28:42.045 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:28:42.844 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T23:28:42.845 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:28:42.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:28:42.845 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T23:28:42.846 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T23:28:42.846 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T23:28:42.852 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T23:28:42.852 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T23:28:50.186 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8158466409731502) > weight (0.25) 2021-02-20T23:28:50.186 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5281649709461144) > weight (0.25) 2021-02-20T23:28:50.187 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8090479406876077) > weight (0.25) 2021-02-20T23:28:50.187 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.46218366698999735) > weight (0.25) 2021-02-20T23:28:50.187 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T23:30:08.854 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.42089853922090226) > weight (0.25) 2021-02-20T23:30:08.866 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T23:30:08.867 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T23:30:08.868 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T23:30:08.869 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T23:30:08.866+0000 7f7c902d7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 81420) UID: 1000 2021-02-20T23:30:09.270 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:30:09.869 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T23:30:09.869 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:30:09.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:30:09.870 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T23:30:09.871 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T23:30:09.871 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T23:30:09.874 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T23:30:09.874 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T23:30:19.188 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T23:30:19.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T23:30:19.193 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T23:30:19.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T23:30:19.190+0000 7fb10768c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 81998) UID: 1000 2021-02-20T23:30:19.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:30:21.195 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T23:30:21.195 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:30:21.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:30:21.196 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T23:30:21.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T23:30:21.197 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T23:30:21.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T23:30:21.200 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T23:31:24.201 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4112254343801385) > weight (0.25) 2021-02-20T23:31:24.209 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4393573853666919) > weight (0.25) 2021-02-20T23:31:24.210 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9106677748114697) > weight (0.25) 2021-02-20T23:31:24.210 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3135184237702834) > weight (0.25) 2021-02-20T23:31:24.211 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T23:31:58.876 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4105525725468787) > weight (0.25) 2021-02-20T23:31:58.876 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3876151853945575) > weight (0.25) 2021-02-20T23:31:58.877 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2999120668335158) > weight (0.25) 2021-02-20T23:31:58.877 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7431683991276798) > weight (0.25) 2021-02-20T23:31:58.878 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T23:33:09.878 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9422565907009632) > weight (0.25) 2021-02-20T23:33:09.887 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.559588683365477) > weight (0.25) 2021-02-20T23:33:09.887 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.30847792799875573) > weight (0.25) 2021-02-20T23:33:09.887 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T23:33:09.888 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T23:33:09.890 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T23:33:09.895 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T23:33:09.885+0000 7f41edab5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 81627) UID: 1000 2021-02-20T23:33:10.103 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:33:13.891 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T23:33:13.899 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:33:13.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:33:13.900 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T23:33:13.900 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T23:33:13.902 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T23:33:13.905 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T23:33:13.905 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T23:33:17.212 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T23:33:17.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T23:33:17.220 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T23:33:17.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T23:33:17.217+0000 7f459b6ae700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 82364) UID: 1000 2021-02-20T23:33:17.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:33:19.221 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T23:33:19.229 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:33:19.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:33:19.229 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T23:33:19.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T23:33:19.231 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T23:33:19.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T23:33:19.234 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T23:34:38.906 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4558158144088382) > weight (0.25) 2021-02-20T23:34:38.913 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.42932490478627583) > weight (0.25) 2021-02-20T23:34:38.913 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5810979307815739) > weight (0.25) 2021-02-20T23:34:38.913 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T23:34:38.914 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T23:34:38.914 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T23:34:38.915 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T23:34:38.911+0000 7ff90d0df700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 82457) UID: 1000 2021-02-20T23:34:39.116 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:34:47.915 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T23:34:47.916 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:34:47.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:34:47.917 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T23:34:47.917 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T23:34:47.918 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T23:34:47.921 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T23:34:47.922 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T23:35:08.234 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9580707208690421) > weight (0.25) 2021-02-20T23:35:08.243 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8328415282613766) > weight (0.25) 2021-02-20T23:35:08.243 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8734575304163683) > weight (0.25) 2021-02-20T23:35:08.244 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.602272056793201) > weight (0.25) 2021-02-20T23:35:08.244 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T23:36:18.245 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4850949582739573) > weight (0.25) 2021-02-20T23:36:18.281 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9717347204592985) > weight (0.25) 2021-02-20T23:36:18.281 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7398113043971333) > weight (0.25) 2021-02-20T23:36:18.282 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T23:36:18.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T23:36:18.284 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T23:36:18.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-20T23:36:18.280+0000 7f098c697700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 81236) UID: 1000 2021-02-20T23:36:18.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:36:21.285 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T23:36:21.286 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:36:21.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:36:21.287 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T23:36:21.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T23:36:21.287 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T23:36:21.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T23:36:21.291 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T23:36:28.923 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T23:36:28.931 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T23:36:28.933 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T23:36:28.934 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T23:36:28.928+0000 7fbffcf32700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 82182) UID: 1000 2021-02-20T23:36:29.133 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:36:37.934 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T23:36:37.941 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:36:37.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:36:37.942 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T23:36:37.943 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T23:36:37.943 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T23:36:37.946 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T23:36:37.946 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T23:37:28.291 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7280663965072133) > weight (0.25) 2021-02-20T23:37:28.299 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T23:37:28.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T23:37:28.300 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T23:37:28.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T23:37:28.298+0000 7ff12f83e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 81811) UID: 1000 2021-02-20T23:37:28.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:37:31.302 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T23:37:31.302 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:37:31.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:37:31.303 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T23:37:31.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T23:37:31.304 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T23:37:31.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T23:37:31.307 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T23:38:35.950 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5692635708395433) > weight (0.25) 2021-02-20T23:38:35.961 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8277189110611258) > weight (0.25) 2021-02-20T23:38:35.961 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5009047457115688) > weight (0.25) 2021-02-20T23:38:35.961 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T23:38:35.963 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T23:38:35.963 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T23:38:35.964 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T23:38:35.960+0000 7fa1f62f2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 82640) UID: 1000 2021-02-20T23:38:36.165 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:38:41.965 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T23:38:41.965 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:38:41.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:38:41.966 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T23:38:41.967 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T23:38:41.967 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T23:38:41.970 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T23:38:41.971 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T23:39:19.308 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T23:39:19.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T23:39:19.317 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T23:39:19.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T23:39:19.315+0000 7f6b69e7f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 82548) UID: 1000 2021-02-20T23:39:19.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:39:21.318 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T23:39:21.318 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:39:21.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:39:21.319 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T23:39:21.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T23:39:21.320 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T23:39:21.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T23:39:21.323 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T23:40:13.972 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9628706673997285) > weight (0.25) 2021-02-20T23:40:13.980 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7913350181213209) > weight (0.25) 2021-02-20T23:40:13.981 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8805620546597024) > weight (0.25) 2021-02-20T23:40:13.981 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T23:40:13.982 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T23:40:13.982 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T23:40:13.983 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T23:40:13.978+0000 7f088d1bc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 83026) UID: 1000 2021-02-20T23:40:14.183 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:40:18.983 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T23:40:18.990 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:40:18.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:40:18.991 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T23:40:18.992 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T23:40:18.992 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T23:40:18.995 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T23:40:18.995 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T23:40:42.323 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6427023218828362) > weight (0.25) 2021-02-20T23:40:42.343 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T23:40:42.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T23:40:42.345 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T23:40:42.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T23:40:42.345+0000 7f8af70ad700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 82926) UID: 1000 2021-02-20T23:40:42.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:40:45.345 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T23:40:45.346 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:40:45.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:40:45.347 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T23:40:45.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T23:40:45.347 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T23:40:45.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T23:40:45.353 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T23:41:55.997 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9588363305606469) > weight (0.25) 2021-02-20T23:41:56.006 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.39460164987086255) > weight (0.25) 2021-02-20T23:41:56.007 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.31976083348896256) > weight (0.25) 2021-02-20T23:41:56.007 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9255109371583048) > weight (0.25) 2021-02-20T23:41:56.007 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T23:42:09.355 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T23:42:09.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T23:42:09.364 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T23:42:09.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T23:42:09.363+0000 7fae32da7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 83117) UID: 1000 2021-02-20T23:42:09.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:42:18.365 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T23:42:18.372 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:42:18.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:42:18.373 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T23:42:18.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T23:42:18.376 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T23:42:18.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T23:42:18.379 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T23:43:32.009 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7744491629311693) > weight (0.25) 2021-02-20T23:43:32.018 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5404366545270112) > weight (0.25) 2021-02-20T23:43:32.018 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5957640284880613) > weight (0.25) 2021-02-20T23:43:32.018 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3373690701843185) > weight (0.25) 2021-02-20T23:43:32.018 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T23:43:35.380 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7747088865629959) > weight (0.25) 2021-02-20T23:43:35.380 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3489324151144618) > weight (0.25) 2021-02-20T23:43:35.381 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7144983939295724) > weight (0.25) 2021-02-20T23:43:35.381 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.49436843731448743) > weight (0.25) 2021-02-20T23:43:35.381 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T23:44:47.383 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T23:44:47.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T23:44:47.402 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T23:44:47.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T23:44:47.399+0000 7f3deb2c2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 83392) UID: 1000 2021-02-20T23:44:47.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:44:54.019 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9053286787121018) > weight (0.25) 2021-02-20T23:44:54.027 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T23:44:54.028 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T23:44:54.030 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T23:44:54.031 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T23:44:54.026+0000 7fe9b8bca700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 82273) UID: 1000 2021-02-20T23:44:54.230 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:44:54.403 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T23:44:54.403 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:44:54.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:44:54.404 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T23:44:54.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T23:44:54.405 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T23:44:54.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T23:44:54.408 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T23:44:58.030 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T23:44:58.031 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:44:58.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:44:58.033 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T23:44:58.033 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T23:44:58.034 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T23:44:58.038 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T23:44:58.039 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T23:45:08.445 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T23:45:08.438+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a391b040 2021-02-20T23:45:08.447 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T23:45:08.442+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a52807d0 2021-02-20T23:45:08.448 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T23:45:08.442+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6abc0a560 2021-02-20T23:45:08.448 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-20T23:45:08.442+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b8a90b10 2021-02-20T23:46:08.412 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6338230474817358) > weight (0.25) 2021-02-20T23:46:08.432 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6178262015890778) > weight (0.25) 2021-02-20T23:46:08.432 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7774766763931401) > weight (0.25) 2021-02-20T23:46:08.432 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3920037544150373) > weight (0.25) 2021-02-20T23:46:08.434 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T23:46:11.040 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9569969731248171) > weight (0.25) 2021-02-20T23:46:11.041 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7910140372424056) > weight (0.25) 2021-02-20T23:46:11.041 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T23:46:11.042 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T23:46:11.042 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T23:46:11.044 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T23:46:11.040+0000 7f6b454a5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 82089) UID: 1000 2021-02-20T23:46:11.445 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:46:20.043 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T23:46:20.044 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:46:20.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:46:20.045 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T23:46:20.045 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T23:46:20.045 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T23:46:20.048 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T23:46:20.049 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T23:47:43.435 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3716801040309814) > weight (0.25) 2021-02-20T23:47:43.445 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7319011191923662) > weight (0.25) 2021-02-20T23:47:43.445 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3475137307344025) > weight (0.25) 2021-02-20T23:47:43.445 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3116648207515401) > weight (0.25) 2021-02-20T23:47:43.445 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T23:48:12.051 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.687975310239389) > weight (0.25) 2021-02-20T23:48:12.058 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8401310102002243) > weight (0.25) 2021-02-20T23:48:12.058 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5634158740381424) > weight (0.25) 2021-02-20T23:48:12.058 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7572702905263176) > weight (0.25) 2021-02-20T23:48:12.058 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T23:48:13.591 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T23:48:13.585+0000 7f4a1bb72700 -1 reset not still connected to 0x55fca2e10220 2021-02-20T23:48:13.591 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T23:48:13.585+0000 7f4a1bb72700 -1 reset not still connected to 0x55fca2e108a0 2021-02-20T23:48:13.591 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T23:48:13.585+0000 7f4a1bb72700 -1 reset not still connected to 0x55fca2e10d80 2021-02-20T23:48:13.592 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-20T23:48:13.585+0000 7f4a1bb72700 -1 reset not still connected to 0x55fca2e11b50 2021-02-20T23:48:52.446 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7715402755857222) > weight (0.25) 2021-02-20T23:48:52.454 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7936760732404452) > weight (0.25) 2021-02-20T23:48:52.455 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T23:48:52.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T23:48:52.456 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T23:48:52.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T23:48:52.456+0000 7f2e7e140700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 81536) UID: 1000 2021-02-20T23:48:52.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:48:57.457 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T23:48:57.458 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:48:57.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:48:57.458 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T23:48:57.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T23:48:57.459 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T23:48:57.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T23:48:57.463 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T23:49:26.059 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8038210568066694) > weight (0.25) 2021-02-20T23:49:26.067 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.28763037394406954) > weight (0.25) 2021-02-20T23:49:26.068 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T23:49:26.068 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T23:49:26.069 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T23:49:26.069 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T23:49:26.067+0000 7ff1489eb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 83671) UID: 1000 2021-02-20T23:49:26.270 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:49:32.070 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T23:49:32.070 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:49:32.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:49:32.071 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T23:49:32.071 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T23:49:32.072 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T23:49:32.075 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T23:49:32.075 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T23:50:40.464 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T23:50:40.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T23:50:40.472 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T23:50:40.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T23:50:40.465+0000 7fa9163f4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 83486) UID: 1000 2021-02-20T23:50:40.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:50:46.076 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8980092386669186) > weight (0.25) 2021-02-20T23:50:46.077 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5116456500153546) > weight (0.25) 2021-02-20T23:50:46.077 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T23:50:46.078 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T23:50:46.078 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T23:50:46.078 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T23:50:46.078 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:50:46.079 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-20T23:50:46.073+0000 7fa235f26700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 83863) UID: 1000 2021-02-20T23:50:46.279 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:50:48.472 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T23:50:48.473 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:50:48.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:50:48.473 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T23:50:48.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T23:50:48.474 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T23:50:48.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T23:50:48.477 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T23:50:52.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:50:52.180 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T23:50:52.180 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T23:50:52.181 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T23:50:52.183 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T23:50:52.184 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T23:52:06.479 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7245324117801326) > weight (0.25) 2021-02-20T23:52:06.487 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.747769858946354) > weight (0.25) 2021-02-20T23:52:06.488 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T23:52:06.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T23:52:06.489 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T23:52:06.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-20T23:52:06.487+0000 7f7e14189700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 83772) UID: 1000 2021-02-20T23:52:06.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:52:14.490 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T23:52:14.493 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:52:14.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:52:14.494 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T23:52:14.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T23:52:14.495 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T23:52:14.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T23:52:14.498 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T23:52:48.184 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T23:52:48.191 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T23:52:48.192 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T23:52:48.193 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-20T23:52:48.186+0000 7f58dea60700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 82834) UID: 1000 2021-02-20T23:52:48.393 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:52:50.193 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T23:52:50.194 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:52:50.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:52:50.194 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T23:52:50.195 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T23:52:50.195 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T23:52:50.198 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T23:52:50.198 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T23:53:47.500 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3121832623501404) > weight (0.25) 2021-02-20T23:53:47.508 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6897557181861615) > weight (0.25) 2021-02-20T23:53:47.508 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5016049311770603) > weight (0.25) 2021-02-20T23:53:47.508 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.49797103332422676) > weight (0.25) 2021-02-20T23:53:47.508 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T23:53:53.199 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.41269452144647045) > weight (0.25) 2021-02-20T23:53:53.199 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5518010566626537) > weight (0.25) 2021-02-20T23:53:53.199 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9402288625683881) > weight (0.25) 2021-02-20T23:53:53.200 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T23:53:53.200 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T23:53:53.201 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T23:53:53.201 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-20T23:53:53.200+0000 7f6a4a21d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 83210) UID: 1000 2021-02-20T23:53:53.402 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:53:56.201 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T23:53:56.202 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:53:56.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:53:56.203 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T23:53:56.203 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T23:53:56.203 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T23:53:56.208 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T23:53:56.208 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T23:55:38.209 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9820375996307903) > weight (0.25) 2021-02-20T23:55:38.225 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T23:55:38.226 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T23:55:38.227 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T23:55:38.228 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T23:55:38.225+0000 7fc3a953c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 83577) UID: 1000 2021-02-20T23:55:38.629 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:55:41.229 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T23:55:41.229 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:55:41.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:55:41.231 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T23:55:41.231 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T23:55:41.232 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T23:55:41.238 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T23:55:41.238 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T23:55:43.510 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8750070773967457) > weight (0.25) 2021-02-20T23:55:43.514 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T23:55:43.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T23:55:43.515 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T23:55:43.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T23:55:43.513+0000 7fb8156cc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 83301) UID: 1000 2021-02-20T23:55:43.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:55:46.516 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T23:55:46.517 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:55:46.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:55:46.517 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T23:55:46.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T23:55:46.518 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T23:55:46.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T23:55:46.521 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T23:56:43.239 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.840690102367858) > weight (0.25) 2021-02-20T23:56:43.248 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T23:56:43.248 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T23:56:43.249 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T23:56:43.250 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-20T23:56:43.247+0000 7fd9b43c3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 84431) UID: 1000 2021-02-20T23:56:43.450 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:56:47.255 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T23:56:47.263 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:56:47.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:56:47.264 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T23:56:47.264 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T23:56:47.264 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T23:56:47.267 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T23:56:47.268 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T23:57:41.522 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7888547661258842) > weight (0.25) 2021-02-20T23:57:41.535 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T23:57:41.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T23:57:41.536 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T23:57:41.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-20T23:57:41.534+0000 7f7e8a8fd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 84522) UID: 1000 2021-02-20T23:57:41.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:57:45.537 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T23:57:45.538 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:57:45.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:57:45.539 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T23:57:45.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T23:57:45.539 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T23:57:45.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T23:57:45.542 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T23:58:02.269 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5414841442070798) > weight (0.25) 2021-02-20T23:58:02.270 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7519624272986235) > weight (0.25) 2021-02-20T23:58:02.270 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2864223780812901) > weight (0.25) 2021-02-20T23:58:02.270 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7448925451922747) > weight (0.25) 2021-02-20T23:58:02.270 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T23:59:39.271 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2572478984337835) > weight (0.25) 2021-02-20T23:59:39.272 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2765266636733681) > weight (0.25) 2021-02-20T23:59:39.273 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.44316613540423844) > weight (0.25) 2021-02-20T23:59:39.273 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.581203001619808) > weight (0.25) 2021-02-20T23:59:39.273 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T23:59:41.545 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T23:59:41.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T23:59:41.547 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T23:59:41.547 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T23:59:41.548 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T23:59:41.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-20T23:59:41.547+0000 7fa51452b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 83964) UID: 1000 2021-02-20T23:59:41.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-20T23:59:47.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T23:59:47.650 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T23:59:47.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T23:59:47.651 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T23:59:47.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T23:59:47.654 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-21T00:00:12.543 INFO:tasks.ceph.cluster1.osd.0.smithi170.stderr:2021-02-21T00:00:12.530+0000 7f184c55a700 -1 reset not still connected to 0x5655121445b0 2021-02-21T00:01:20.655 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3996656768414323) > weight (0.25) 2021-02-21T00:01:20.694 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5959153066069443) > weight (0.25) 2021-02-21T00:01:20.694 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-21T00:01:20.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-21T00:01:20.696 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-21T00:01:20.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-21T00:01:20.692+0000 7f7ea5d28700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 84154) UID: 1000 2021-02-21T00:01:20.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:01:22.697 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-21T00:01:22.698 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:01:22.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:01:22.700 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-21T00:01:22.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-21T00:01:22.701 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-21T00:01:22.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-21T00:01:22.709 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-21T00:01:37.274 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5158435317046951) > weight (0.25) 2021-02-21T00:01:37.281 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5878839665371045) > weight (0.25) 2021-02-21T00:01:37.281 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9931272732028141) > weight (0.25) 2021-02-21T00:01:37.281 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-21T00:01:37.282 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-21T00:01:37.283 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-21T00:01:37.283 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-21T00:01:37.283 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:01:37.284 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-21T00:01:37.280+0000 7f7bb087a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 84337) UID: 1000 2021-02-21T00:01:37.484 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:01:43.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:01:43.393 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-21T00:01:43.394 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-21T00:01:43.394 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-21T00:01:43.397 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-21T00:01:43.398 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-21T00:02:56.399 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3298269189435882) > weight (0.25) 2021-02-21T00:02:56.408 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7560672174845015) > weight (0.25) 2021-02-21T00:02:56.408 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.47743336894058197) > weight (0.25) 2021-02-21T00:02:56.409 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.45860455553219215) > weight (0.25) 2021-02-21T00:02:56.409 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-21T00:02:59.710 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.548418269914245) > weight (0.25) 2021-02-21T00:02:59.711 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.848965320490991) > weight (0.25) 2021-02-21T00:02:59.711 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-21T00:02:59.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-21T00:02:59.712 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-21T00:02:59.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-21T00:02:59.709+0000 7fdaf4999700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 84962) UID: 1000 2021-02-21T00:02:59.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:03:02.713 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-21T00:03:02.714 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:03:02.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:03:02.715 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-21T00:03:02.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-21T00:03:02.716 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-21T00:03:02.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-21T00:03:02.719 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-21T00:04:12.728 INFO:tasks.ceph.cluster1.osd.0.smithi170.stderr:2021-02-21T00:04:12.720+0000 7f184c55a700 -1 reset not still connected to 0x5655121445b0 2021-02-21T00:04:15.409 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-21T00:04:15.418 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-21T00:04:15.419 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-21T00:04:15.420 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-21T00:04:15.415+0000 7fdc2db33700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 84245) UID: 1000 2021-02-21T00:04:15.623 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:04:15.720 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.27245272698868306) > weight (0.25) 2021-02-21T00:04:15.720 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6905784250532102) > weight (0.25) 2021-02-21T00:04:15.720 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4226802768810052) > weight (0.25) 2021-02-21T00:04:15.721 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5613717683332121) > weight (0.25) 2021-02-21T00:04:15.721 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-21T00:04:24.419 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-21T00:04:24.427 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:04:24.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:04:24.429 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-21T00:04:24.429 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-21T00:04:24.430 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-21T00:04:24.434 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-21T00:04:24.435 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-21T00:05:29.435 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-21T00:05:29.444 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-21T00:05:29.445 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-21T00:05:29.451 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-21T00:05:29.441+0000 7fcf60cb1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 85237) UID: 1000 2021-02-21T00:05:29.649 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:05:32.446 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-21T00:05:32.446 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:05:32.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:05:32.448 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-21T00:05:32.448 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-21T00:05:32.449 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-21T00:05:32.453 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-21T00:05:32.453 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-21T00:05:58.722 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5644531209362884) > weight (0.25) 2021-02-21T00:05:58.729 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8726928976263841) > weight (0.25) 2021-02-21T00:05:58.729 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-21T00:05:58.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-21T00:05:58.731 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-21T00:05:58.731 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-21T00:05:58.731 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:05:58.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-21T00:05:58.725+0000 7f1785c6d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 85145) UID: 1000 2021-02-21T00:05:58.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:06:04.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:06:04.842 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-21T00:06:04.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-21T00:06:04.843 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-21T00:06:04.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-21T00:06:04.848 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-21T00:07:04.849 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4443634214192256) > weight (0.25) 2021-02-21T00:07:04.858 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-21T00:07:04.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-21T00:07:04.860 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-21T00:07:04.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:07:04.855+0000 7f2a9819e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 84706) UID: 1000 2021-02-21T00:07:05.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:07:05.861 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-21T00:07:05.862 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:07:05.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:07:05.862 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-21T00:07:05.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-21T00:07:05.863 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-21T00:07:05.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-21T00:07:05.867 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-21T00:07:09.454 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9077349641746724) > weight (0.25) 2021-02-21T00:07:09.461 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6651390794206593) > weight (0.25) 2021-02-21T00:07:09.461 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.46322997880707917) > weight (0.25) 2021-02-21T00:07:09.461 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8120123361369176) > weight (0.25) 2021-02-21T00:07:09.461 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-21T00:08:34.463 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.612928367069067) > weight (0.25) 2021-02-21T00:08:34.473 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2880051369610158) > weight (0.25) 2021-02-21T00:08:34.473 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9453423277895474) > weight (0.25) 2021-02-21T00:08:34.473 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.32094330064912524) > weight (0.25) 2021-02-21T00:08:34.474 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-21T00:08:57.868 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9346179505043405) > weight (0.25) 2021-02-21T00:08:57.869 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-21T00:08:57.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-21T00:08:57.871 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-21T00:08:57.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:08:57.868+0000 7f0e0ac59700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 85514) UID: 1000 2021-02-21T00:08:58.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:09:04.871 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-21T00:09:04.872 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:09:04.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:09:04.873 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-21T00:09:04.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-21T00:09:04.874 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-21T00:09:04.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-21T00:09:04.877 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-21T00:10:10.475 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9260812641803361) > weight (0.25) 2021-02-21T00:10:10.482 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-21T00:10:10.482 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-21T00:10:10.483 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-21T00:10:10.484 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-21T00:10:10.482+0000 7f4feb6ce700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 84615) UID: 1000 2021-02-21T00:10:10.522 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-21T00:10:10.514+0000 7f03ccd38700 -1 reset not still connected to 0x556b8794d110 2021-02-21T00:10:10.684 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:10:18.484 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-21T00:10:18.484 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:10:18.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:10:18.485 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-21T00:10:18.486 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-21T00:10:18.486 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-21T00:10:18.489 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-21T00:10:18.489 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-21T00:10:35.878 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5221987577489035) > weight (0.25) 2021-02-21T00:10:35.884 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4443335450012429) > weight (0.25) 2021-02-21T00:10:35.885 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5102047470914846) > weight (0.25) 2021-02-21T00:10:35.885 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-21T00:10:35.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-21T00:10:35.886 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-21T00:10:35.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-21T00:10:35.885+0000 7f05e4b7a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 82735) UID: 1000 2021-02-21T00:10:36.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:10:40.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:10:40.241+0000 7fcf4eca6700 -1 rbd::mirror::ImageReplayer: 0x55bfa85fb400 [3/2464eeeb-6954-426d-a8e5-129943cea55c] start: already running 2021-02-21T00:10:40.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:10:40.241+0000 7fcf4eca6700 -1 rbd::mirror::ImageReplayer: 0x55bfa85fb680 [3/3ab5666f-763c-442d-bc17-ea037dcc3a42] start: already running 2021-02-21T00:10:40.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:10:40.241+0000 7fcf4eca6700 -1 rbd::mirror::ImageReplayer: 0x55bfa85fb900 [3/3c6940ee-a642-422e-aafb-e3920bba87cf] start: already running 2021-02-21T00:10:40.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:10:40.245+0000 7fcf4eca6700 -1 rbd::mirror::ImageReplayer: 0x55bfa85fbb80 [3/3d38caf8-6420-4f62-95dc-628caa5144da] start: already running 2021-02-21T00:10:40.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:10:40.245+0000 7fcf4eca6700 -1 rbd::mirror::ImageReplayer: 0x55bfa85fbe00 [3/49a0b5eb-7d69-4824-806b-1ece12354eb9] start: already running 2021-02-21T00:10:40.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:10:40.245+0000 7fcf4eca6700 -1 rbd::mirror::ImageReplayer: 0x55bfa85fc080 [3/4c862af6-d1da-4215-b6b8-461f3883b104] start: already running 2021-02-21T00:10:40.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:10:40.245+0000 7fcf4eca6700 -1 rbd::mirror::ImageReplayer: 0x55bfa85fc300 [3/4cba354e-99d2-4859-a4d0-baeec2203656] start: already running 2021-02-21T00:10:40.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:10:40.245+0000 7fcf4eca6700 -1 rbd::mirror::ImageReplayer: 0x55bfa85fc580 [3/567ace7d-44e4-4d42-b47a-38bd35ec0e11] start: already running 2021-02-21T00:10:40.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:10:40.245+0000 7fcf4eca6700 -1 rbd::mirror::ImageReplayer: 0x55bfa85fc800 [3/5786ec48-0da5-470d-b6a6-ddd71a666657] start: already running 2021-02-21T00:10:40.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:10:40.245+0000 7fcf4eca6700 -1 rbd::mirror::ImageReplayer: 0x55bfa85fca80 [3/5c036645-b242-4fa8-bde0-d5b28dda3fd3] start: already running 2021-02-21T00:10:40.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:10:40.245+0000 7fcf4eca6700 -1 rbd::mirror::ImageReplayer: 0x55bfa85fcd00 [3/5f3204b5-90c6-4244-a940-e749860457d5] start: already running 2021-02-21T00:10:40.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:10:40.245+0000 7fcf4eca6700 -1 rbd::mirror::ImageReplayer: 0x55bfa85fcf80 [3/675a1b7b-4e17-4b1e-9e92-eaf406f1c33f] start: already running 2021-02-21T00:10:40.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:10:40.245+0000 7fcf4eca6700 -1 rbd::mirror::ImageReplayer: 0x55bfa85fd200 [3/681dbea5-7557-4dba-a92d-01da85613e31] start: already running 2021-02-21T00:10:40.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:10:40.245+0000 7fcf4eca6700 -1 rbd::mirror::ImageReplayer: 0x55bfa85fd480 [3/753c14e7-97d4-4109-8814-2c7f1abbbee8] start: already running 2021-02-21T00:10:43.886 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-21T00:10:43.893 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:10:43.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:10:43.894 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-21T00:10:43.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-21T00:10:43.895 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-21T00:10:43.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-21T00:10:43.898 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-21T00:12:12.492 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7360828984401168) > weight (0.25) 2021-02-21T00:12:12.507 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5273163252032903) > weight (0.25) 2021-02-21T00:12:12.507 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.984525940920134) > weight (0.25) 2021-02-21T00:12:12.507 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3237426704161419) > weight (0.25) 2021-02-21T00:12:12.507 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-21T00:12:38.901 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9410209146284041) > weight (0.25) 2021-02-21T00:12:38.910 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5113316321381762) > weight (0.25) 2021-02-21T00:12:38.910 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-21T00:12:38.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-21T00:12:38.912 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-21T00:12:38.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-21T00:12:38.910+0000 7f285c6bc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 85422) UID: 1000 2021-02-21T00:12:39.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:12:46.913 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-21T00:12:46.920 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:12:46.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:12:46.921 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-21T00:12:46.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-21T00:12:46.921 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-21T00:12:46.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-21T00:12:46.925 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-21T00:13:43.509 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-21T00:13:43.513 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-21T00:13:43.514 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-21T00:13:43.514 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-21T00:13:43.512+0000 7f2712ee1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 85331) UID: 1000 2021-02-21T00:13:43.715 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:13:48.515 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-21T00:13:48.515 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:13:48.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:13:48.516 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-21T00:13:48.517 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-21T00:13:48.517 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-21T00:13:48.520 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-21T00:13:48.521 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-21T00:14:22.926 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.35801123777101673) > weight (0.25) 2021-02-21T00:14:22.927 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-21T00:14:22.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-21T00:14:22.928 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-21T00:14:22.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:14:22.927+0000 7f87ea5f9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 85630) UID: 1000 2021-02-21T00:14:23.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:14:25.930 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-21T00:14:25.930 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:14:25.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:14:25.932 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-21T00:14:25.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-21T00:14:25.933 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-21T00:14:25.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-21T00:14:25.939 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-21T00:15:00.522 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-21T00:15:00.523 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-21T00:15:00.523 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-21T00:15:00.524 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-21T00:15:00.522+0000 7fb6aa526700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 86006) UID: 1000 2021-02-21T00:15:00.725 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:15:07.523 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-21T00:15:07.524 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:15:07.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:15:07.525 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-21T00:15:07.526 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-21T00:15:07.527 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-21T00:15:07.530 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-21T00:15:07.531 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-21T00:16:04.943 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-21T00:16:04.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-21T00:16:04.945 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-21T00:16:04.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:16:04.945+0000 7fcf4fca8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 84869) UID: 1000 2021-02-21T00:16:05.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:16:08.531 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7384581623032629) > weight (0.25) 2021-02-21T00:16:08.532 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9634094126566748) > weight (0.25) 2021-02-21T00:16:08.532 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9505469352528131) > weight (0.25) 2021-02-21T00:16:08.532 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7500346090594003) > weight (0.25) 2021-02-21T00:16:08.533 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-21T00:16:09.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:16:09.320+0000 7feb8bfa5700 -1 rbd::mirror::ImageReplayer: 0x560b54bdc300 [3/49a0b5eb-7d69-4824-806b-1ece12354eb9] start: already running 2021-02-21T00:16:09.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:16:09.324+0000 7feb8bfa5700 -1 rbd::mirror::ImageReplayer: 0x560b58b43e00 [3/972a582c-01df-4309-8926-a24b908ec0fc] start: already running 2021-02-21T00:16:09.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:16:09.324+0000 7feb8bfa5700 -1 rbd::mirror::ImageReplayer: 0x560b58b44080 [3/9940675f-5679-4d59-9726-da9895d6bb7c] start: already running 2021-02-21T00:16:09.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:16:09.324+0000 7feb8bfa5700 -1 rbd::mirror::ImageReplayer: 0x560b58b44300 [3/9b42b7b2-0767-46b8-a6a6-e0b14c0863eb] start: already running 2021-02-21T00:16:09.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:16:09.324+0000 7feb8b7a4700 -1 rbd::mirror::ImageReplayer: 0x560b58b44580 [3/9bb53f74-14d3-4057-8f64-9bba63f946e2] start: already running 2021-02-21T00:16:09.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:16:09.324+0000 7feb8b7a4700 -1 rbd::mirror::ImageReplayer: 0x560b58b44800 [3/a3cf9f38-0520-4671-b928-6d907b15b2fb] start: already running 2021-02-21T00:16:09.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:16:09.324+0000 7feb8bfa5700 -1 rbd::mirror::ImageReplayer: 0x560b58b44a80 [3/a3d3aab0-3900-425e-a825-80fa9ff092ea] start: already running 2021-02-21T00:16:09.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:16:09.324+0000 7feb8bfa5700 -1 rbd::mirror::ImageReplayer: 0x560b58b44d00 [3/ab28f848-782d-4c9d-a26a-5bdb09404113] start: already running 2021-02-21T00:16:09.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:16:09.324+0000 7feb8b7a4700 -1 rbd::mirror::ImageReplayer: 0x560b58b44f80 [3/b1b702c0-1f24-4c9b-b9bc-cc3a0752ca38] start: already running 2021-02-21T00:16:09.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:16:09.324+0000 7feb8bfa5700 -1 rbd::mirror::ImageReplayer: 0x560b58b45200 [3/b97c3e7d-cef7-4890-a315-ffa1f7e51804] start: already running 2021-02-21T00:16:09.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:16:09.324+0000 7feb8bfa5700 -1 rbd::mirror::ImageReplayer: 0x560b58b45480 [3/bbf1cffa-5e93-4c1c-937d-71c8c54806ff] start: already running 2021-02-21T00:16:09.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:16:09.324+0000 7feb8bfa5700 -1 rbd::mirror::ImageReplayer: 0x560b58b45700 [3/cdea041d-e1fe-4928-9842-3ce9553bc980] start: already running 2021-02-21T00:16:09.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:16:09.324+0000 7feb8b7a4700 -1 rbd::mirror::ImageReplayer: 0x560b58b45980 [3/cdfd4b53-cfc9-488d-957e-07aa3d636acf] start: already running 2021-02-21T00:16:09.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:16:09.324+0000 7feb8b7a4700 -1 rbd::mirror::ImageReplayer: 0x560b58b45c00 [3/ce32d773-a477-4402-af84-407be17c35f5] start: already running 2021-02-21T00:16:09.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:16:09.324+0000 7feb8b7a4700 -1 rbd::mirror::ImageReplayer: 0x560b58bea000 [3/dd0c591d-efef-4cf8-9976-7731785fbcd4] start: already running 2021-02-21T00:16:09.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:16:09.864+0000 7feb8bfa5700 -1 rbd::mirror::ImageReplayer: 0x560b58b45c00 [3/ce32d773-a477-4402-af84-407be17c35f5] register_admin_socket_hook: error registering admin socket commands 2021-02-21T00:16:09.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:16:09.864+0000 7feb8bfa5700 -1 rbd::mirror::ImageReplayer: 0x560b54bdb680 [3/205dce0d-bc7c-41b6-8ed9-87e6a63d1a70] register_admin_socket_hook: error registering admin socket commands 2021-02-21T00:16:10.949 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-21T00:16:10.949 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:16:10.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:16:10.950 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-21T00:16:10.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-21T00:16:10.951 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-21T00:16:10.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-21T00:16:10.956 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-21T00:16:52.482 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-21T00:16:52.474+0000 7fb8259e4700 -1 reset not still connected to 0x557860c14220 2021-02-21T00:16:52.482 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-21T00:16:52.474+0000 7fb8259e4700 -1 reset not still connected to 0x557863951e10 2021-02-21T00:16:52.485 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-21T00:16:52.478+0000 7fb8259e4700 -1 reset not still connected to 0x5578639538e0 2021-02-21T00:17:38.534 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6306255149798884) > weight (0.25) 2021-02-21T00:17:38.535 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6229658663842363) > weight (0.25) 2021-02-21T00:17:38.535 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8178573000442944) > weight (0.25) 2021-02-21T00:17:38.535 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-21T00:17:38.536 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-21T00:17:38.537 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-21T00:17:38.538 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-21T00:17:38.534+0000 7f5eda756700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 85053) UID: 1000 2021-02-21T00:17:38.738 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:17:46.537 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-21T00:17:46.538 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:17:46.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:17:46.539 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-21T00:17:46.539 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-21T00:17:46.539 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-21T00:17:46.542 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-21T00:17:46.543 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-21T00:17:58.605 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-21T00:17:58.601+0000 7fb8259e4700 -1 reset not still connected to 0x557863951e10 2021-02-21T00:17:58.607 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-21T00:17:58.605+0000 7fb8259e4700 -1 reset not still connected to 0x5578639538e0 2021-02-21T00:18:06.956 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6521162327911421) > weight (0.25) 2021-02-21T00:18:06.957 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.39650984946427936) > weight (0.25) 2021-02-21T00:18:06.957 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8483918648995263) > weight (0.25) 2021-02-21T00:18:06.957 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8935848323146975) > weight (0.25) 2021-02-21T00:18:06.957 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-21T00:19:19.544 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-21T00:19:19.545 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-21T00:19:19.546 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-21T00:19:19.547 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-21T00:19:19.543+0000 7fb4d0423700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 86192) UID: 1000 2021-02-21T00:19:19.747 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:19:21.546 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-21T00:19:21.547 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:19:21.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:19:21.548 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-21T00:19:21.548 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-21T00:19:21.549 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-21T00:19:21.551 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-21T00:19:21.552 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-21T00:19:30.959 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.940607818591145) > weight (0.25) 2021-02-21T00:19:31.414 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-21T00:19:31.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-21T00:19:31.415 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-21T00:19:31.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:19:31.419+0000 7feb8c7a6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 86098) UID: 1000 2021-02-21T00:19:31.476 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-21T00:19:31.471+0000 7f03ccd38700 -1 reset not still connected to 0x556b80daf810 2021-02-21T00:19:31.476 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-21T00:19:31.471+0000 7f03ccd38700 -1 reset not still connected to 0x556b82600f70 2021-02-21T00:19:31.476 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-21T00:19:31.471+0000 7f03ccd38700 -1 reset not still connected to 0x556b826028a0 2021-02-21T00:19:31.477 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-21T00:19:31.471+0000 7f03ccd38700 -1 reset not still connected to 0x556b8f7bf5f0 2021-02-21T00:19:31.478 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-21T00:19:31.471+0000 7f03ccd38700 -1 reset not still connected to 0x556b8f7c0630 2021-02-21T00:19:31.480 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-21T00:19:31.471+0000 7f03ccd38700 -1 reset not still connected to 0x556b9f07d790 2021-02-21T00:19:31.482 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-21T00:19:31.475+0000 7f03ccd38700 -1 reset not still connected to 0x556b9f90c9c0 2021-02-21T00:19:31.483 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-21T00:19:31.475+0000 7f03ccd38700 -1 reset not still connected to 0x556b9f90dc70 2021-02-21T00:19:31.483 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-21T00:19:31.475+0000 7f03ccd38700 -1 reset not still connected to 0x556b9f90e8a0 2021-02-21T00:19:31.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:19:35.416 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-21T00:19:35.417 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:19:35.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:19:35.418 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-21T00:19:35.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-21T00:19:35.418 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-21T00:19:35.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-21T00:19:35.421 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-21T00:21:00.422 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-21T00:21:00.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-21T00:21:00.424 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-21T00:21:00.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:21:00.417+0000 7f5057683700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 86291) UID: 1000 2021-02-21T00:21:00.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:21:05.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-21T00:21:05.269+0000 7facf5f0f700 -1 rbd::mirror::ImageReplayer: 0x5646d28d6300 [3/9bb53f74-14d3-4057-8f64-9bba63f946e2] register_admin_socket_hook: error registering admin socket commands 2021-02-21T00:21:05.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-21T00:21:05.269+0000 7facf570e700 -1 rbd::mirror::ImageReplayer: 0x5646d28d6300 [3/9bb53f74-14d3-4057-8f64-9bba63f946e2] register_admin_socket_hook: error registering admin socket commands 2021-02-21T00:21:05.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-21T00:21:05.269+0000 7facf5f0f700 -1 rbd::mirror::ImageReplayer: 0x5646d291ef00 [3/ff38d3a1-22f6-4a7c-a814-2d4505cf6570] register_admin_socket_hook: error registering admin socket commands 2021-02-21T00:21:05.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-21T00:21:05.269+0000 7facf570e700 -1 rbd::mirror::ImageReplayer: 0x5646d291ef00 [3/ff38d3a1-22f6-4a7c-a814-2d4505cf6570] register_admin_socket_hook: error registering admin socket commands 2021-02-21T00:21:05.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-21T00:21:05.269+0000 7facf5f0f700 -1 rbd::mirror::ImageReplayer: 0x5646d291e280 [3/dd0f679e-1316-4b67-b5c5-1d6037eb2c90] register_admin_socket_hook: error registering admin socket commands 2021-02-21T00:21:05.425 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-21T00:21:05.426 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:21:05.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:21:05.426 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-21T00:21:05.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-21T00:21:05.427 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-21T00:21:05.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-21T00:21:05.430 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-21T00:21:08.553 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.35610423488892895) > weight (0.25) 2021-02-21T00:21:08.554 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-21T00:21:08.554 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-21T00:21:08.554 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-21T00:21:08.555 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-21T00:21:08.552+0000 7f7b9c167700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 85722) UID: 1000 2021-02-21T00:21:08.756 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:21:16.555 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-21T00:21:16.555 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:21:16.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:21:16.556 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-21T00:21:16.556 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-21T00:21:16.557 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-21T00:21:16.559 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-21T00:21:16.560 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-21T00:22:34.430 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5297000663861514) > weight (0.25) 2021-02-21T00:22:34.431 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.39200549493850745) > weight (0.25) 2021-02-21T00:22:34.431 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-21T00:22:34.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-21T00:22:34.433 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-21T00:22:34.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-21T00:22:34.430+0000 7f87d3c7d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 85914) UID: 1000 2021-02-21T00:22:34.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:22:35.433 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-21T00:22:35.433 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:22:35.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:22:35.434 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-21T00:22:35.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-21T00:22:35.435 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-21T00:22:35.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-21T00:22:35.438 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-21T00:23:02.561 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-21T00:23:02.563 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-21T00:23:02.563 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-21T00:23:02.564 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-21T00:23:02.557+0000 7f0012c26700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 86479) UID: 1000 2021-02-21T00:23:02.765 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:23:09.565 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-21T00:23:09.565 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:23:09.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:23:09.566 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-21T00:23:09.566 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-21T00:23:09.567 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-21T00:23:09.570 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-21T00:23:09.570 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-21T00:24:10.439 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9541660331825097) > weight (0.25) 2021-02-21T00:24:10.440 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5140837652824751) > weight (0.25) 2021-02-21T00:24:10.440 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6356937888797414) > weight (0.25) 2021-02-21T00:24:10.441 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.952730537773582) > weight (0.25) 2021-02-21T00:24:10.441 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-21T00:24:24.572 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6592280208536212) > weight (0.25) 2021-02-21T00:24:24.573 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7448859403073469) > weight (0.25) 2021-02-21T00:24:24.573 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.47843047748628786) > weight (0.25) 2021-02-21T00:24:24.573 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.479836879412358) > weight (0.25) 2021-02-21T00:24:24.573 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-21T00:25:42.442 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8973643039278069) > weight (0.25) 2021-02-21T00:25:42.455 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4811663046416552) > weight (0.25) 2021-02-21T00:25:42.456 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3557267207122856) > weight (0.25) 2021-02-21T00:25:42.456 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6161461128011294) > weight (0.25) 2021-02-21T00:25:42.456 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-21T00:26:19.575 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6302385631981782) > weight (0.25) 2021-02-21T00:26:19.582 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.466212771656199) > weight (0.25) 2021-02-21T00:26:19.582 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7816642463211244) > weight (0.25) 2021-02-21T00:26:19.582 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5765101192271437) > weight (0.25) 2021-02-21T00:26:19.583 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-21T00:27:20.457 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8743501067572034) > weight (0.25) 2021-02-21T00:27:20.465 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-21T00:27:20.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-21T00:27:20.467 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-21T00:27:20.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:27:20.463+0000 7f2d7b662700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 86570) UID: 1000 2021-02-21T00:27:20.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:27:25.468 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-21T00:27:25.468 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:27:25.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:27:25.469 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-21T00:27:25.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-21T00:27:25.470 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-21T00:27:25.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-21T00:27:25.475 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-21T00:27:47.584 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.33555201299038717) > weight (0.25) 2021-02-21T00:27:47.585 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.26942904040090887) > weight (0.25) 2021-02-21T00:27:47.585 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6177288055841041) > weight (0.25) 2021-02-21T00:27:47.585 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-21T00:27:47.586 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-21T00:27:47.586 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-21T00:27:47.587 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-21T00:27:47.587 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:27:47.587 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-21T00:27:47.582+0000 7f661418e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 86387) UID: 1000 2021-02-21T00:27:47.787 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:27:53.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:27:53.690 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-21T00:27:53.690 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-21T00:27:53.690 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-21T00:27:53.693 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-21T00:27:53.693 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-21T00:29:06.477 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3796100299242211) > weight (0.25) 2021-02-21T00:29:06.478 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-21T00:29:06.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-21T00:29:06.479 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-21T00:29:06.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:29:06.476+0000 7efdee862700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 87048) UID: 1000 2021-02-21T00:29:06.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:29:08.480 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-21T00:29:08.481 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:29:08.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:29:08.482 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-21T00:29:08.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-21T00:29:08.483 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-21T00:29:08.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-21T00:29:08.487 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-21T00:29:51.694 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5482397146564382) > weight (0.25) 2021-02-21T00:29:51.694 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-21T00:29:51.695 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-21T00:29:51.696 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-21T00:29:51.696 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-21T00:29:51.691+0000 7f57d0056700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 86769) UID: 1000 2021-02-21T00:29:51.897 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:29:56.696 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-21T00:29:56.697 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:29:56.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:29:56.698 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-21T00:29:56.698 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-21T00:29:56.698 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-21T00:29:56.701 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-21T00:29:56.702 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-21T00:30:26.489 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6567911093328922) > weight (0.25) 2021-02-21T00:30:26.489 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.876177696603243) > weight (0.25) 2021-02-21T00:30:26.490 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.851176284301332) > weight (0.25) 2021-02-21T00:30:26.490 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7137779757875394) > weight (0.25) 2021-02-21T00:30:26.490 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-21T00:31:37.491 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7698148223846294) > weight (0.25) 2021-02-21T00:31:37.492 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9922895603445031) > weight (0.25) 2021-02-21T00:31:37.492 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.39521517558231645) > weight (0.25) 2021-02-21T00:31:37.492 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-21T00:31:37.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-21T00:31:37.494 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-21T00:31:37.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-21T00:31:37.496+0000 7facf6710700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 85821) UID: 1000 2021-02-21T00:31:37.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:31:40.702 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-21T00:31:40.703 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-21T00:31:40.703 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-21T00:31:40.704 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-21T00:31:40.704 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:31:40.705 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-21T00:31:40.700+0000 7fc5a294a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 86951) UID: 1000 2021-02-21T00:31:40.905 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:31:41.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.856+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d900 [3/2464eeeb-6954-426d-a8e5-129943cea55c] start: already running 2021-02-21T00:31:41.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.856+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81db80 [3/3ab5666f-763c-442d-bc17-ea037dcc3a42] start: already running 2021-02-21T00:31:41.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.856+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81de00 [3/3c6940ee-a642-422e-aafb-e3920bba87cf] start: already running 2021-02-21T00:31:41.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.904+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81fc00 [3/7658e9e0-5116-4c47-bad5-4e7894e27da7] start: already running 2021-02-21T00:31:41.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.904+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d180 [3/172baefb-5ce5-45e0-997e-249560de6fbd] start: already running 2021-02-21T00:31:41.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.904+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d400 [3/1a170226-518b-4a75-85a8-6fbdc9b5d9f0] start: already running 2021-02-21T00:31:41.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.904+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d680 [3/205dce0d-bc7c-41b6-8ed9-87e6a63d1a70] start: already running 2021-02-21T00:31:41.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.904+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d900 [3/2464eeeb-6954-426d-a8e5-129943cea55c] start: already running 2021-02-21T00:31:41.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.904+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81db80 [3/3ab5666f-763c-442d-bc17-ea037dcc3a42] start: already running 2021-02-21T00:31:41.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.904+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81de00 [3/3c6940ee-a642-422e-aafb-e3920bba87cf] start: already running 2021-02-21T00:31:41.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.904+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81e080 [3/3d38caf8-6420-4f62-95dc-628caa5144da] start: already running 2021-02-21T00:31:41.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.904+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81e580 [3/4c862af6-d1da-4215-b6b8-461f3883b104] start: already running 2021-02-21T00:31:41.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81e800 [3/4cba354e-99d2-4859-a4d0-baeec2203656] start: already running 2021-02-21T00:31:41.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81ea80 [3/567ace7d-44e4-4d42-b47a-38bd35ec0e11] start: already running 2021-02-21T00:31:41.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81ed00 [3/5786ec48-0da5-470d-b6a6-ddd71a666657] start: already running 2021-02-21T00:31:41.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81ef80 [3/5c036645-b242-4fa8-bde0-d5b28dda3fd3] start: already running 2021-02-21T00:31:41.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81f200 [3/5f3204b5-90c6-4244-a940-e749860457d5] start: already running 2021-02-21T00:31:41.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81f480 [3/675a1b7b-4e17-4b1e-9e92-eaf406f1c33f] start: already running 2021-02-21T00:31:41.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81f700 [3/681dbea5-7557-4dba-a92d-01da85613e31] start: already running 2021-02-21T00:31:41.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81f980 [3/753c14e7-97d4-4109-8814-2c7f1abbbee8] start: already running 2021-02-21T00:31:41.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4000 [3/790c038c-4353-4ff1-ae0b-9c9bb1e1adaa] start: already running 2021-02-21T00:31:41.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4280 [3/7fb27511-6f76-444f-b338-191b2c7caa61] start: already running 2021-02-21T00:31:41.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4500 [3/858c7e91-92b9-4e8b-8ae9-f8cb46e242bf] start: already running 2021-02-21T00:31:41.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4780 [3/87eed4f8-1d97-46e0-b439-8f6b85b99b40] start: already running 2021-02-21T00:31:41.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4a00 [3/8c216315-5117-4846-b850-c4265b4697a3] start: already running 2021-02-21T00:31:41.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4c80 [3/91672d77-7c82-4d00-b945-306681ce602c] start: already running 2021-02-21T00:31:41.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4f00 [3/941323f6-0453-4944-ab79-664fde6ec580] start: already running 2021-02-21T00:31:41.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5180 [3/945695ad-0f15-4fd7-a092-4e7df53e1d1b] start: already running 2021-02-21T00:31:41.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5400 [3/972a582c-01df-4309-8926-a24b908ec0fc] start: already running 2021-02-21T00:31:41.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5680 [3/9940675f-5679-4d59-9726-da9895d6bb7c] start: already running 2021-02-21T00:31:41.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5900 [3/9b42b7b2-0767-46b8-a6a6-e0b14c0863eb] start: already running 2021-02-21T00:31:41.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5b80 [3/9bb53f74-14d3-4057-8f64-9bba63f946e2] start: already running 2021-02-21T00:31:41.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6080 [3/a3cf9f38-0520-4671-b928-6d907b15b2fb] start: already running 2021-02-21T00:31:41.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d180 [3/172baefb-5ce5-45e0-997e-249560de6fbd] start: already running 2021-02-21T00:31:41.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d400 [3/1a170226-518b-4a75-85a8-6fbdc9b5d9f0] start: already running 2021-02-21T00:31:41.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d680 [3/205dce0d-bc7c-41b6-8ed9-87e6a63d1a70] start: already running 2021-02-21T00:31:41.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d900 [3/2464eeeb-6954-426d-a8e5-129943cea55c] start: already running 2021-02-21T00:31:41.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81db80 [3/3ab5666f-763c-442d-bc17-ea037dcc3a42] start: already running 2021-02-21T00:31:41.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81de00 [3/3c6940ee-a642-422e-aafb-e3920bba87cf] start: already running 2021-02-21T00:31:41.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81e080 [3/3d38caf8-6420-4f62-95dc-628caa5144da] start: already running 2021-02-21T00:31:41.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81e580 [3/4c862af6-d1da-4215-b6b8-461f3883b104] start: already running 2021-02-21T00:31:41.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81e800 [3/4cba354e-99d2-4859-a4d0-baeec2203656] start: already running 2021-02-21T00:31:41.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81ea80 [3/567ace7d-44e4-4d42-b47a-38bd35ec0e11] start: already running 2021-02-21T00:31:41.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81ed00 [3/5786ec48-0da5-470d-b6a6-ddd71a666657] start: already running 2021-02-21T00:31:41.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81ef80 [3/5c036645-b242-4fa8-bde0-d5b28dda3fd3] start: already running 2021-02-21T00:31:41.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81f200 [3/5f3204b5-90c6-4244-a940-e749860457d5] start: already running 2021-02-21T00:31:41.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81f480 [3/675a1b7b-4e17-4b1e-9e92-eaf406f1c33f] start: already running 2021-02-21T00:31:41.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81f700 [3/681dbea5-7557-4dba-a92d-01da85613e31] start: already running 2021-02-21T00:31:41.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81f980 [3/753c14e7-97d4-4109-8814-2c7f1abbbee8] start: already running 2021-02-21T00:31:41.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81fc00 [3/7658e9e0-5116-4c47-bad5-4e7894e27da7] start: already running 2021-02-21T00:31:41.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4000 [3/790c038c-4353-4ff1-ae0b-9c9bb1e1adaa] start: already running 2021-02-21T00:31:41.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d180 [3/172baefb-5ce5-45e0-997e-249560de6fbd] start: already running 2021-02-21T00:31:41.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.908+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d400 [3/1a170226-518b-4a75-85a8-6fbdc9b5d9f0] start: already running 2021-02-21T00:31:41.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81e580 [3/4c862af6-d1da-4215-b6b8-461f3883b104] start: already running 2021-02-21T00:31:41.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81e800 [3/4cba354e-99d2-4859-a4d0-baeec2203656] start: already running 2021-02-21T00:31:41.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81ea80 [3/567ace7d-44e4-4d42-b47a-38bd35ec0e11] start: already running 2021-02-21T00:31:41.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81ed00 [3/5786ec48-0da5-470d-b6a6-ddd71a666657] start: already running 2021-02-21T00:31:41.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81ef80 [3/5c036645-b242-4fa8-bde0-d5b28dda3fd3] start: already running 2021-02-21T00:31:41.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81f200 [3/5f3204b5-90c6-4244-a940-e749860457d5] start: already running 2021-02-21T00:31:41.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81f480 [3/675a1b7b-4e17-4b1e-9e92-eaf406f1c33f] start: already running 2021-02-21T00:31:41.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81f700 [3/681dbea5-7557-4dba-a92d-01da85613e31] start: already running 2021-02-21T00:31:41.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81f980 [3/753c14e7-97d4-4109-8814-2c7f1abbbee8] start: already running 2021-02-21T00:31:41.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81fc00 [3/7658e9e0-5116-4c47-bad5-4e7894e27da7] start: already running 2021-02-21T00:31:41.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4000 [3/790c038c-4353-4ff1-ae0b-9c9bb1e1adaa] start: already running 2021-02-21T00:31:41.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4280 [3/7fb27511-6f76-444f-b338-191b2c7caa61] start: already running 2021-02-21T00:31:41.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4500 [3/858c7e91-92b9-4e8b-8ae9-f8cb46e242bf] start: already running 2021-02-21T00:31:41.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4780 [3/87eed4f8-1d97-46e0-b439-8f6b85b99b40] start: already running 2021-02-21T00:31:41.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4a00 [3/8c216315-5117-4846-b850-c4265b4697a3] start: already running 2021-02-21T00:31:41.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4c80 [3/91672d77-7c82-4d00-b945-306681ce602c] start: already running 2021-02-21T00:31:41.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4f00 [3/941323f6-0453-4944-ab79-664fde6ec580] start: already running 2021-02-21T00:31:41.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5180 [3/945695ad-0f15-4fd7-a092-4e7df53e1d1b] start: already running 2021-02-21T00:31:41.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5400 [3/972a582c-01df-4309-8926-a24b908ec0fc] start: already running 2021-02-21T00:31:41.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5680 [3/9940675f-5679-4d59-9726-da9895d6bb7c] start: already running 2021-02-21T00:31:41.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5900 [3/9b42b7b2-0767-46b8-a6a6-e0b14c0863eb] start: already running 2021-02-21T00:31:41.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5b80 [3/9bb53f74-14d3-4057-8f64-9bba63f946e2] start: already running 2021-02-21T00:31:41.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6080 [3/a3cf9f38-0520-4671-b928-6d907b15b2fb] start: already running 2021-02-21T00:31:41.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6300 [3/a3d3aab0-3900-425e-a825-80fa9ff092ea] start: already running 2021-02-21T00:31:41.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5e00 [3/ab28f848-782d-4c9d-a26a-5bdb09404113] start: already running 2021-02-21T00:31:41.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6580 [3/b1b702c0-1f24-4c9b-b9bc-cc3a0752ca38] start: already running 2021-02-21T00:31:41.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6800 [3/b97c3e7d-cef7-4890-a315-ffa1f7e51804] start: already running 2021-02-21T00:31:41.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6a80 [3/bbf1cffa-5e93-4c1c-937d-71c8c54806ff] start: already running 2021-02-21T00:31:41.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6d00 [3/cdea041d-e1fe-4928-9842-3ce9553bc980] start: already running 2021-02-21T00:31:41.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6f80 [3/cdfd4b53-cfc9-488d-957e-07aa3d636acf] start: already running 2021-02-21T00:31:41.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d7200 [3/ce32d773-a477-4402-af84-407be17c35f5] start: already running 2021-02-21T00:31:41.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d7480 [3/dd0c591d-efef-4cf8-9976-7731785fbcd4] start: already running 2021-02-21T00:31:41.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d7700 [3/dd0f679e-1316-4b67-b5c5-1d6037eb2c90] start: already running 2021-02-21T00:31:41.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d7980 [3/def81481-a32e-45b7-8067-6c32798fc165] start: already running 2021-02-21T00:31:41.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d7c00 [3/e5f43e4f-211a-45bf-bf8b-2e986b46e688] start: already running 2021-02-21T00:31:41.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d25388a000 [3/eafa574f-c2c8-4659-b559-be75ec7d56bb] start: already running 2021-02-21T00:31:41.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d25388a280 [3/ef164021-65e4-434b-a5d7-bcdd53df22fd] start: already running 2021-02-21T00:31:41.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d25388a500 [3/ff38d3a1-22f6-4a7c-a814-2d4505cf6570] start: already running 2021-02-21T00:31:41.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.912+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d25388a780 [3/ff64bb69-23c4-4b3a-9b65-a48c78854429] start: already running 2021-02-21T00:31:41.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d680 [3/205dce0d-bc7c-41b6-8ed9-87e6a63d1a70] start: already running 2021-02-21T00:31:41.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d900 [3/2464eeeb-6954-426d-a8e5-129943cea55c] start: already running 2021-02-21T00:31:41.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81db80 [3/3ab5666f-763c-442d-bc17-ea037dcc3a42] start: already running 2021-02-21T00:31:41.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81de00 [3/3c6940ee-a642-422e-aafb-e3920bba87cf] start: already running 2021-02-21T00:31:41.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81e080 [3/3d38caf8-6420-4f62-95dc-628caa5144da] start: already running 2021-02-21T00:31:41.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81e300 [3/49a0b5eb-7d69-4824-806b-1ece12354eb9] start: already running 2021-02-21T00:31:41.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d180 [3/172baefb-5ce5-45e0-997e-249560de6fbd] start: already running 2021-02-21T00:31:41.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d400 [3/1a170226-518b-4a75-85a8-6fbdc9b5d9f0] start: already running 2021-02-21T00:31:41.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d680 [3/205dce0d-bc7c-41b6-8ed9-87e6a63d1a70] start: already running 2021-02-21T00:31:41.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d900 [3/2464eeeb-6954-426d-a8e5-129943cea55c] start: already running 2021-02-21T00:31:41.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81db80 [3/3ab5666f-763c-442d-bc17-ea037dcc3a42] start: already running 2021-02-21T00:31:41.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81de00 [3/3c6940ee-a642-422e-aafb-e3920bba87cf] start: already running 2021-02-21T00:31:41.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81e080 [3/3d38caf8-6420-4f62-95dc-628caa5144da] start: already running 2021-02-21T00:31:41.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81e300 [3/49a0b5eb-7d69-4824-806b-1ece12354eb9] start: already running 2021-02-21T00:31:41.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81e580 [3/4c862af6-d1da-4215-b6b8-461f3883b104] start: already running 2021-02-21T00:31:41.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81e800 [3/4cba354e-99d2-4859-a4d0-baeec2203656] start: already running 2021-02-21T00:31:41.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81ea80 [3/567ace7d-44e4-4d42-b47a-38bd35ec0e11] start: already running 2021-02-21T00:31:41.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81ed00 [3/5786ec48-0da5-470d-b6a6-ddd71a666657] start: already running 2021-02-21T00:31:41.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81ef80 [3/5c036645-b242-4fa8-bde0-d5b28dda3fd3] start: already running 2021-02-21T00:31:41.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81f200 [3/5f3204b5-90c6-4244-a940-e749860457d5] start: already running 2021-02-21T00:31:41.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81f480 [3/675a1b7b-4e17-4b1e-9e92-eaf406f1c33f] start: already running 2021-02-21T00:31:41.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81f700 [3/681dbea5-7557-4dba-a92d-01da85613e31] start: already running 2021-02-21T00:31:41.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81f980 [3/753c14e7-97d4-4109-8814-2c7f1abbbee8] start: already running 2021-02-21T00:31:41.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81fc00 [3/7658e9e0-5116-4c47-bad5-4e7894e27da7] start: already running 2021-02-21T00:31:41.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4000 [3/790c038c-4353-4ff1-ae0b-9c9bb1e1adaa] start: already running 2021-02-21T00:31:41.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4280 [3/7fb27511-6f76-444f-b338-191b2c7caa61] start: already running 2021-02-21T00:31:41.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4500 [3/858c7e91-92b9-4e8b-8ae9-f8cb46e242bf] start: already running 2021-02-21T00:31:41.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4780 [3/87eed4f8-1d97-46e0-b439-8f6b85b99b40] start: already running 2021-02-21T00:31:41.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4a00 [3/8c216315-5117-4846-b850-c4265b4697a3] start: already running 2021-02-21T00:31:41.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4c80 [3/91672d77-7c82-4d00-b945-306681ce602c] start: already running 2021-02-21T00:31:41.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4f00 [3/941323f6-0453-4944-ab79-664fde6ec580] start: already running 2021-02-21T00:31:41.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5180 [3/945695ad-0f15-4fd7-a092-4e7df53e1d1b] start: already running 2021-02-21T00:31:41.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5400 [3/972a582c-01df-4309-8926-a24b908ec0fc] start: already running 2021-02-21T00:31:41.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5680 [3/9940675f-5679-4d59-9726-da9895d6bb7c] start: already running 2021-02-21T00:31:41.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5900 [3/9b42b7b2-0767-46b8-a6a6-e0b14c0863eb] start: already running 2021-02-21T00:31:41.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5b80 [3/9bb53f74-14d3-4057-8f64-9bba63f946e2] start: already running 2021-02-21T00:31:41.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6080 [3/a3cf9f38-0520-4671-b928-6d907b15b2fb] start: already running 2021-02-21T00:31:41.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6300 [3/a3d3aab0-3900-425e-a825-80fa9ff092ea] start: already running 2021-02-21T00:31:41.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5e00 [3/ab28f848-782d-4c9d-a26a-5bdb09404113] start: already running 2021-02-21T00:31:41.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6580 [3/b1b702c0-1f24-4c9b-b9bc-cc3a0752ca38] start: already running 2021-02-21T00:31:41.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6800 [3/b97c3e7d-cef7-4890-a315-ffa1f7e51804] start: already running 2021-02-21T00:31:41.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6a80 [3/bbf1cffa-5e93-4c1c-937d-71c8c54806ff] start: already running 2021-02-21T00:31:41.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6d00 [3/cdea041d-e1fe-4928-9842-3ce9553bc980] start: already running 2021-02-21T00:31:41.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6f80 [3/cdfd4b53-cfc9-488d-957e-07aa3d636acf] start: already running 2021-02-21T00:31:41.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d7200 [3/ce32d773-a477-4402-af84-407be17c35f5] start: already running 2021-02-21T00:31:41.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d7480 [3/dd0c591d-efef-4cf8-9976-7731785fbcd4] start: already running 2021-02-21T00:31:41.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d7700 [3/dd0f679e-1316-4b67-b5c5-1d6037eb2c90] start: already running 2021-02-21T00:31:41.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d7980 [3/def81481-a32e-45b7-8067-6c32798fc165] start: already running 2021-02-21T00:31:41.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.960+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d7c00 [3/e5f43e4f-211a-45bf-bf8b-2e986b46e688] start: already running 2021-02-21T00:31:41.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.964+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d25388a000 [3/eafa574f-c2c8-4659-b559-be75ec7d56bb] start: already running 2021-02-21T00:31:41.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d25388a280 [3/ef164021-65e4-434b-a5d7-bcdd53df22fd] start: already running 2021-02-21T00:31:41.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d25388a500 [3/ff38d3a1-22f6-4a7c-a814-2d4505cf6570] start: already running 2021-02-21T00:31:41.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d25388a780 [3/ff64bb69-23c4-4b3a-9b65-a48c78854429] start: already running 2021-02-21T00:31:41.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d180 [3/172baefb-5ce5-45e0-997e-249560de6fbd] start: already running 2021-02-21T00:31:41.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d400 [3/1a170226-518b-4a75-85a8-6fbdc9b5d9f0] start: already running 2021-02-21T00:31:41.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d680 [3/205dce0d-bc7c-41b6-8ed9-87e6a63d1a70] start: already running 2021-02-21T00:31:41.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d900 [3/2464eeeb-6954-426d-a8e5-129943cea55c] start: already running 2021-02-21T00:31:41.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81db80 [3/3ab5666f-763c-442d-bc17-ea037dcc3a42] start: already running 2021-02-21T00:31:41.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81de00 [3/3c6940ee-a642-422e-aafb-e3920bba87cf] start: already running 2021-02-21T00:31:41.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81e080 [3/3d38caf8-6420-4f62-95dc-628caa5144da] start: already running 2021-02-21T00:31:41.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81e580 [3/4c862af6-d1da-4215-b6b8-461f3883b104] start: already running 2021-02-21T00:31:41.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81e800 [3/4cba354e-99d2-4859-a4d0-baeec2203656] start: already running 2021-02-21T00:31:41.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81ea80 [3/567ace7d-44e4-4d42-b47a-38bd35ec0e11] start: already running 2021-02-21T00:31:41.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81ed00 [3/5786ec48-0da5-470d-b6a6-ddd71a666657] start: already running 2021-02-21T00:31:41.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81ef80 [3/5c036645-b242-4fa8-bde0-d5b28dda3fd3] start: already running 2021-02-21T00:31:41.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81f200 [3/5f3204b5-90c6-4244-a940-e749860457d5] start: already running 2021-02-21T00:31:41.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81f480 [3/675a1b7b-4e17-4b1e-9e92-eaf406f1c33f] start: already running 2021-02-21T00:31:41.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81f700 [3/681dbea5-7557-4dba-a92d-01da85613e31] start: already running 2021-02-21T00:31:41.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81f980 [3/753c14e7-97d4-4109-8814-2c7f1abbbee8] start: already running 2021-02-21T00:31:41.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81fc00 [3/7658e9e0-5116-4c47-bad5-4e7894e27da7] start: already running 2021-02-21T00:31:41.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4000 [3/790c038c-4353-4ff1-ae0b-9c9bb1e1adaa] start: already running 2021-02-21T00:31:41.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4280 [3/7fb27511-6f76-444f-b338-191b2c7caa61] start: already running 2021-02-21T00:31:41.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4500 [3/858c7e91-92b9-4e8b-8ae9-f8cb46e242bf] start: already running 2021-02-21T00:31:41.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.988+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4780 [3/87eed4f8-1d97-46e0-b439-8f6b85b99b40] start: already running 2021-02-21T00:31:41.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.992+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4a00 [3/8c216315-5117-4846-b850-c4265b4697a3] start: already running 2021-02-21T00:31:41.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.992+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4c80 [3/91672d77-7c82-4d00-b945-306681ce602c] start: already running 2021-02-21T00:31:41.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.992+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d4f00 [3/941323f6-0453-4944-ab79-664fde6ec580] start: already running 2021-02-21T00:31:41.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.992+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5180 [3/945695ad-0f15-4fd7-a092-4e7df53e1d1b] start: already running 2021-02-21T00:31:41.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.992+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5400 [3/972a582c-01df-4309-8926-a24b908ec0fc] start: already running 2021-02-21T00:31:41.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.992+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5680 [3/9940675f-5679-4d59-9726-da9895d6bb7c] start: already running 2021-02-21T00:31:41.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.992+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5900 [3/9b42b7b2-0767-46b8-a6a6-e0b14c0863eb] start: already running 2021-02-21T00:31:41.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.992+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5b80 [3/9bb53f74-14d3-4057-8f64-9bba63f946e2] start: already running 2021-02-21T00:31:42.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.992+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6080 [3/a3cf9f38-0520-4671-b928-6d907b15b2fb] start: already running 2021-02-21T00:31:42.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.992+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6300 [3/a3d3aab0-3900-425e-a825-80fa9ff092ea] start: already running 2021-02-21T00:31:42.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.992+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d5e00 [3/ab28f848-782d-4c9d-a26a-5bdb09404113] start: already running 2021-02-21T00:31:42.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.992+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6580 [3/b1b702c0-1f24-4c9b-b9bc-cc3a0752ca38] start: already running 2021-02-21T00:31:42.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.992+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6800 [3/b97c3e7d-cef7-4890-a315-ffa1f7e51804] start: already running 2021-02-21T00:31:42.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.992+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6a80 [3/bbf1cffa-5e93-4c1c-937d-71c8c54806ff] start: already running 2021-02-21T00:31:42.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.992+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6d00 [3/cdea041d-e1fe-4928-9842-3ce9553bc980] start: already running 2021-02-21T00:31:42.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.992+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d6f80 [3/cdfd4b53-cfc9-488d-957e-07aa3d636acf] start: already running 2021-02-21T00:31:42.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.992+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d7200 [3/ce32d773-a477-4402-af84-407be17c35f5] start: already running 2021-02-21T00:31:42.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.992+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d7480 [3/dd0c591d-efef-4cf8-9976-7731785fbcd4] start: already running 2021-02-21T00:31:42.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.992+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d7700 [3/dd0f679e-1316-4b67-b5c5-1d6037eb2c90] start: already running 2021-02-21T00:31:42.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.992+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d2538d7980 [3/def81481-a32e-45b7-8067-6c32798fc165] start: already running 2021-02-21T00:31:42.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.992+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d2538d7c00 [3/e5f43e4f-211a-45bf-bf8b-2e986b46e688] start: already running 2021-02-21T00:31:42.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.996+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d25388a000 [3/eafa574f-c2c8-4659-b559-be75ec7d56bb] start: already running 2021-02-21T00:31:42.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.996+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d25388a280 [3/ef164021-65e4-434b-a5d7-bcdd53df22fd] start: already running 2021-02-21T00:31:42.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.996+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d25388a500 [3/ff38d3a1-22f6-4a7c-a814-2d4505cf6570] start: already running 2021-02-21T00:31:42.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:41.996+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d25388a780 [3/ff64bb69-23c4-4b3a-9b65-a48c78854429] start: already running 2021-02-21T00:31:42.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:42.156+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81e580 [3/4c862af6-d1da-4215-b6b8-461f3883b104] register_admin_socket_hook: error registering admin socket commands 2021-02-21T00:31:42.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:42.156+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81e580 [3/4c862af6-d1da-4215-b6b8-461f3883b104] register_admin_socket_hook: error registering admin socket commands 2021-02-21T00:31:42.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:42.156+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81ef80 [3/5c036645-b242-4fa8-bde0-d5b28dda3fd3] register_admin_socket_hook: error registering admin socket commands 2021-02-21T00:31:42.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:42.156+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81ef80 [3/5c036645-b242-4fa8-bde0-d5b28dda3fd3] register_admin_socket_hook: error registering admin socket commands 2021-02-21T00:31:42.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:42.156+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81e080 [3/3d38caf8-6420-4f62-95dc-628caa5144da] register_admin_socket_hook: error registering admin socket commands 2021-02-21T00:31:42.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:42.156+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81e080 [3/3d38caf8-6420-4f62-95dc-628caa5144da] register_admin_socket_hook: error registering admin socket commands 2021-02-21T00:31:42.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:42.156+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d400 [3/1a170226-518b-4a75-85a8-6fbdc9b5d9f0] register_admin_socket_hook: error registering admin socket commands 2021-02-21T00:31:42.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:42.156+0000 7f4c87f81700 -1 rbd::mirror::ImageReplayer: 0x55d24f81d400 [3/1a170226-518b-4a75-85a8-6fbdc9b5d9f0] register_admin_socket_hook: error registering admin socket commands 2021-02-21T00:31:42.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:31:42.156+0000 7f4c88782700 -1 rbd::mirror::ImageReplayer: 0x55d24f81f200 [3/5f3204b5-90c6-4244-a940-e749860457d5] register_admin_socket_hook: error registering admin socket commands 2021-02-21T00:31:45.494 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-21T00:31:45.495 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:31:45.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:31:45.496 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-21T00:31:45.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-21T00:31:45.496 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-21T00:31:45.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-21T00:31:45.499 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-21T00:31:46.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:31:46.807 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-21T00:31:46.807 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-21T00:31:46.807 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-21T00:31:46.811 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-21T00:31:46.812 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-21T00:33:00.813 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2549143363202755) > weight (0.25) 2021-02-21T00:33:00.814 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.46739008097140633) > weight (0.25) 2021-02-21T00:33:00.814 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5291308708938173) > weight (0.25) 2021-02-21T00:33:00.815 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.735563284284762) > weight (0.25) 2021-02-21T00:33:00.815 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-21T00:33:05.501 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-21T00:33:05.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-21T00:33:05.502 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-21T00:33:05.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:33:05.497+0000 7f4c88f83700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 86678) UID: 1000 2021-02-21T00:33:05.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:33:12.503 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-21T00:33:12.504 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:33:12.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:33:12.505 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-21T00:33:12.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-21T00:33:12.505 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-21T00:33:12.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-21T00:33:12.511 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-21T00:33:22.468 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-21T00:33:22.461+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a44b09c0 2021-02-21T00:33:22.469 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-21T00:33:22.461+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b5aa55a0 2021-02-21T00:33:22.469 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-21T00:33:22.461+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6bab4f740 2021-02-21T00:33:22.469 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-21T00:33:22.461+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6e58aab60 2021-02-21T00:33:22.469 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-21T00:33:22.461+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6e58aac30 2021-02-21T00:34:21.816 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7523897026660697) > weight (0.25) 2021-02-21T00:34:21.817 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.44460436385863367) > weight (0.25) 2021-02-21T00:34:21.817 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3721687894803547) > weight (0.25) 2021-02-21T00:34:21.817 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8447948588192161) > weight (0.25) 2021-02-21T00:34:21.818 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-21T00:34:43.512 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-21T00:34:43.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-21T00:34:43.514 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-21T00:34:43.514 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-21T00:34:43.514 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:34:43.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:34:43.511+0000 7f0003553700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 87613) UID: 1000 2021-02-21T00:34:43.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:34:49.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:34:49.617 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-21T00:34:49.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-21T00:34:49.618 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-21T00:34:49.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-21T00:34:49.622 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-21T00:35:23.819 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-21T00:35:23.820 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-21T00:35:23.820 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-21T00:35:23.821 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-21T00:35:23.814+0000 7f47a315f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 87514) UID: 1000 2021-02-21T00:35:24.022 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:35:32.821 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-21T00:35:32.821 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:35:32.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:35:32.822 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-21T00:35:32.823 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-21T00:35:32.823 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-21T00:35:32.826 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-21T00:35:32.826 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-21T00:36:08.624 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-21T00:36:08.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-21T00:36:08.625 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-21T00:36:08.626 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-21T00:36:08.626 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:36:08.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:36:08.625+0000 7fd3a995b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 87706) UID: 1000 2021-02-21T00:36:08.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:36:14.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:36:14.807 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-21T00:36:14.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-21T00:36:14.808 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-21T00:36:14.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-21T00:36:14.818 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-21T00:37:21.819 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4837598194747732) > weight (0.25) 2021-02-21T00:37:21.819 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4234123465758075) > weight (0.25) 2021-02-21T00:37:21.820 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-21T00:37:21.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-21T00:37:21.821 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-21T00:37:21.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-21T00:37:21.819+0000 7fa83cabf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 86860) UID: 1000 2021-02-21T00:37:21.827 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.43835562465330635) > weight (0.25) 2021-02-21T00:37:21.827 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4637536208544394) > weight (0.25) 2021-02-21T00:37:21.828 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3882672600473238) > weight (0.25) 2021-02-21T00:37:21.828 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.29619198546033265) > weight (0.25) 2021-02-21T00:37:21.828 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-21T00:37:22.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:37:26.822 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-21T00:37:26.822 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:37:26.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:37:26.823 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-21T00:37:26.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-21T00:37:26.824 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-21T00:37:26.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-21T00:37:26.828 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-21T00:37:35.928 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-21T00:37:35.922+0000 7f4a1bb72700 -1 reset not still connected to 0x55fc7a3155a0 2021-02-21T00:37:35.931 INFO:tasks.ceph.cluster2.osd.0.smithi183.stderr:2021-02-21T00:37:35.922+0000 7f4a1bb72700 -1 reset not still connected to 0x55fc8e5fde90 2021-02-21T00:38:35.829 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8946483407467842) > weight (0.25) 2021-02-21T00:38:35.838 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3266930834887114) > weight (0.25) 2021-02-21T00:38:35.838 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7560143701892073) > weight (0.25) 2021-02-21T00:38:35.838 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-21T00:38:35.839 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-21T00:38:35.839 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-21T00:38:35.840 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-21T00:38:35.840 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:38:35.841 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-21T00:38:35.837+0000 7f53e5cfe700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 87139) UID: 1000 2021-02-21T00:38:36.040 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:38:38.829 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-21T00:38:38.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-21T00:38:38.835 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-21T00:38:38.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:38:38.833+0000 7ffb8d365700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 87891) UID: 1000 2021-02-21T00:38:39.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:38:41.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:38:41.949 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-21T00:38:41.949 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-21T00:38:41.950 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-21T00:38:41.952 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-21T00:38:41.953 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-21T00:38:42.836 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-21T00:38:42.836 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:38:42.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:38:42.837 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-21T00:38:42.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-21T00:38:42.838 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-21T00:38:42.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-21T00:38:42.841 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-21T00:39:22.462 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-21T00:39:22.459+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a44b09c0 2021-02-21T00:39:22.463 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-21T00:39:22.459+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b5aa55a0 2021-02-21T00:39:22.463 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-21T00:39:22.459+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6bab4f740 2021-02-21T00:39:22.463 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-21T00:39:22.459+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6e58aab60 2021-02-21T00:39:22.464 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-21T00:39:22.459+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6e58aac30 2021-02-21T00:40:10.842 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8095045765422372) > weight (0.25) 2021-02-21T00:40:10.843 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7283364557008141) > weight (0.25) 2021-02-21T00:40:10.844 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7155996852231359) > weight (0.25) 2021-02-21T00:40:10.844 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7796955389789446) > weight (0.25) 2021-02-21T00:40:10.844 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-21T00:40:21.953 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2950939083629375) > weight (0.25) 2021-02-21T00:40:21.954 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-21T00:40:21.955 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-21T00:40:21.955 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-21T00:40:21.956 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-21T00:40:21.954+0000 7f08d8567700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 87323) UID: 1000 2021-02-21T00:40:22.156 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:40:29.956 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-21T00:40:29.957 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:40:29.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:40:29.958 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-21T00:40:29.958 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-21T00:40:29.959 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-21T00:40:29.962 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-21T00:40:29.962 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-21T00:41:47.845 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.678874339828423) > weight (0.25) 2021-02-21T00:41:47.846 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5950843672140426) > weight (0.25) 2021-02-21T00:41:47.846 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-21T00:41:47.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-21T00:41:47.848 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-21T00:41:47.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-21T00:41:47.844+0000 7f0fb2376700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 87983) UID: 1000 2021-02-21T00:41:48.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:41:51.848 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-21T00:41:51.849 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:41:51.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:41:51.850 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-21T00:41:51.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-21T00:41:51.851 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-21T00:41:51.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-21T00:41:51.854 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-21T00:41:55.963 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.46136349518447506) > weight (0.25) 2021-02-21T00:41:55.963 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-21T00:41:55.964 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-21T00:41:55.964 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-21T00:41:55.965 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-21T00:41:55.959+0000 7f4807f92700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 88260) UID: 1000 2021-02-21T00:41:56.165 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:41:58.964 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-21T00:41:58.965 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:41:58.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:41:58.965 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-21T00:41:58.966 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-21T00:41:58.966 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-21T00:41:58.969 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-21T00:41:58.970 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-21T00:42:52.494 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-21T00:42:52.485+0000 7fb8259e4700 -1 reset not still connected to 0x557860c14220 2021-02-21T00:42:52.495 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-21T00:42:52.485+0000 7fb8259e4700 -1 reset not still connected to 0x557863951e10 2021-02-21T00:42:52.495 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-21T00:42:52.489+0000 7fb8259e4700 -1 reset not still connected to 0x5578639538e0 2021-02-21T00:43:00.971 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8551493537327574) > weight (0.25) 2021-02-21T00:43:00.971 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7829972863588134) > weight (0.25) 2021-02-21T00:43:00.972 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5311297716227342) > weight (0.25) 2021-02-21T00:43:00.972 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8566972314164982) > weight (0.25) 2021-02-21T00:43:00.972 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-21T00:43:01.855 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6893896028080296) > weight (0.25) 2021-02-21T00:43:01.856 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7003837751051579) > weight (0.25) 2021-02-21T00:43:01.856 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-21T00:43:01.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-21T00:43:01.857 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-21T00:43:01.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-21T00:43:01.854+0000 7f8d4714f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 88351) UID: 1000 2021-02-21T00:43:02.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:43:04.858 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-21T00:43:04.859 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:43:04.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:43:04.859 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-21T00:43:04.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-21T00:43:04.860 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-21T00:43:04.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-21T00:43:04.863 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-21T00:44:22.973 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9318590127260974) > weight (0.25) 2021-02-21T00:44:22.975 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-21T00:44:22.975 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-21T00:44:22.976 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-21T00:44:22.977 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-21T00:44:22.971+0000 7f27350cf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 88442) UID: 1000 2021-02-21T00:44:23.177 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:44:28.977 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-21T00:44:28.978 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:44:28.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:44:28.979 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-21T00:44:28.979 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-21T00:44:28.979 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-21T00:44:28.982 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-21T00:44:28.982 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-21T00:44:46.864 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9437795504674248) > weight (0.25) 2021-02-21T00:44:46.864 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-21T00:44:46.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-21T00:44:46.865 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-21T00:44:46.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:44:46.863+0000 7f18d6e75700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 87231) UID: 1000 2021-02-21T00:44:47.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:44:51.866 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-21T00:44:51.867 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:44:51.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:44:51.867 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-21T00:44:51.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-21T00:44:51.868 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-21T00:44:51.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-21T00:44:51.872 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-21T00:46:25.983 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5802427154614275) > weight (0.25) 2021-02-21T00:46:25.984 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-21T00:46:25.985 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-21T00:46:25.985 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-21T00:46:25.986 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:2021-02-21T00:46:25.984+0000 7f6b09e93700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 88625) UID: 1000 2021-02-21T00:46:26.187 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:46:27.986 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-21T00:46:27.987 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:46:27.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:46:27.988 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-21T00:46:27.988 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-21T00:46:27.988 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-21T00:46:27.991 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-21T00:46:27.991 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-21T00:46:48.872 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5006194737206937) > weight (0.25) 2021-02-21T00:46:48.873 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6004048581906629) > weight (0.25) 2021-02-21T00:46:48.873 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-21T00:46:48.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-21T00:46:48.874 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-21T00:46:48.874 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-21T00:46:48.875 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:46:48.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:2021-02-21T00:46:48.871+0000 7f8211a61700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 88534) UID: 1000 2021-02-21T00:46:49.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:46:54.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:46:54.977 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-21T00:46:54.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-21T00:46:54.978 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-21T00:46:54.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-21T00:46:54.982 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-21T00:47:57.992 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6610276848199775) > weight (0.25) 2021-02-21T00:47:57.993 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9409836889797746) > weight (0.25) 2021-02-21T00:47:57.993 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-21T00:47:57.994 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-21T00:47:57.994 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-21T00:47:57.996 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:2021-02-21T00:47:57.994+0000 7f8f6afca700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 84055) UID: 1000 2021-02-21T00:47:58.196 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:48:01.996 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-21T00:48:01.996 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:48:01.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:48:01.997 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-21T00:48:01.997 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-21T00:48:01.998 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-21T00:48:02.000 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-21T00:48:02.001 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-21T00:48:22.983 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3875326925220196) > weight (0.25) 2021-02-21T00:48:22.983 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-21T00:48:22.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-21T00:48:22.984 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-21T00:48:22.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:48:22.977+0000 7ff38aae9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 88717) UID: 1000 2021-02-21T00:48:23.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:48:28.985 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-21T00:48:28.986 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:48:28.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:48:28.987 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-21T00:48:28.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-21T00:48:28.988 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-21T00:48:28.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-21T00:48:28.992 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-21T00:49:38.002 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6472529114143297) > weight (0.25) 2021-02-21T00:49:38.003 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.38791541696874077) > weight (0.25) 2021-02-21T00:49:38.004 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6987096554131821) > weight (0.25) 2021-02-21T00:49:38.004 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-21T00:49:38.005 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-21T00:49:38.006 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-21T00:49:38.007 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:2021-02-21T00:49:38.003+0000 7f76eec53700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 88075) UID: 1000 2021-02-21T00:49:38.207 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:49:42.994 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-21T00:49:42.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-21T00:49:42.995 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-21T00:49:42.995 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-21T00:49:42.995 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:49:42.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:49:42.991+0000 7ff6c5f14700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 88166) UID: 1000 2021-02-21T00:49:43.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:49:46.007 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-21T00:49:46.008 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:49:46.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:49:46.009 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-21T00:49:46.009 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-21T00:49:46.009 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-21T00:49:46.012 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-21T00:49:46.012 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-21T00:49:49.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:49:49.097 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-21T00:49:49.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-21T00:49:49.098 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-21T00:49:49.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-21T00:49:49.101 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-21T00:51:43.101 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6761645269260705) > weight (0.25) 2021-02-21T00:51:43.102 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9298241973630881) > weight (0.25) 2021-02-21T00:51:43.102 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.742451063948825) > weight (0.25) 2021-02-21T00:51:43.103 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-21T00:51:43.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-21T00:51:43.104 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-21T00:51:43.104 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-21T00:51:43.104 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:51:43.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:2021-02-21T00:51:43.100+0000 7f2a7e28a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 87423) UID: 1000 2021-02-21T00:51:43.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:51:44.013 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-21T00:51:44.014 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-21T00:51:44.014 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-21T00:51:44.015 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-21T00:51:44.012+0000 7fdcc6e7d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 87800) UID: 1000 2021-02-21T00:51:44.215 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:51:47.699 INFO:tasks.ceph.cluster1.osd.0.smithi170.stderr:2021-02-21T00:51:47.692+0000 7f184c55a700 -1 reset not still connected to 0x56550b8cd810 2021-02-21T00:51:47.699 INFO:tasks.ceph.cluster1.osd.0.smithi170.stderr:2021-02-21T00:51:47.692+0000 7f184c55a700 -1 reset not still connected to 0x56551a14d6c0 2021-02-21T00:51:47.875 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-21T00:51:47.864+0000 7fb8259e4700 -1 reset not still connected to 0x55785d8283c0 2021-02-21T00:51:47.875 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-21T00:51:47.864+0000 7fb8259e4700 -1 reset not still connected to 0x55785d8290c0 2021-02-21T00:51:47.876 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-21T00:51:47.864+0000 7fb8259e4700 -1 reset not still connected to 0x557863951930 2021-02-21T00:51:47.876 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-21T00:51:47.864+0000 7fb8259e4700 -1 reset not still connected to 0x557863951ad0 2021-02-21T00:51:47.876 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-21T00:51:47.864+0000 7fb8259e4700 -1 reset not still connected to 0x557863952150 2021-02-21T00:51:47.876 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-21T00:51:47.864+0000 7fb8259e4700 -1 reset not still connected to 0x5578639530c0 2021-02-21T00:51:47.877 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-21T00:51:47.864+0000 7fb8259e4700 -1 reset not still connected to 0x5578639534d0 2021-02-21T00:51:47.878 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-21T00:51:47.864+0000 7fb8259e4700 -1 reset not still connected to 0x557866b049c0 2021-02-21T00:51:47.878 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-21T00:51:47.864+0000 7fb8259e4700 -1 reset not still connected to 0x557866b06220 2021-02-21T00:51:47.878 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-21T00:51:47.864+0000 7fb8259e4700 -1 reset not still connected to 0x557866b06630 2021-02-21T00:51:47.878 INFO:tasks.ceph.cluster1.osd.2.smithi170.stderr:2021-02-21T00:51:47.864+0000 7fb8259e4700 -1 reset not still connected to 0x557866b079b0 2021-02-21T00:51:49.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:51:49.222 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-21T00:51:49.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-21T00:51:49.223 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-21T00:51:49.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-21T00:51:49.227 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-21T00:51:50.015 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-21T00:51:50.015 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:51:50.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:51:50.016 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-21T00:51:50.016 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-21T00:51:50.016 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-21T00:51:50.023 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-21T00:51:50.023 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-21T00:53:27.024 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.650725585901922) > weight (0.25) 2021-02-21T00:53:27.025 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8389244090277864) > weight (0.25) 2021-02-21T00:53:27.025 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.748780830059644) > weight (0.25) 2021-02-21T00:53:27.025 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9909780492776168) > weight (0.25) 2021-02-21T00:53:27.025 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-21T00:53:42.228 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7640606474774291) > weight (0.25) 2021-02-21T00:53:42.229 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9572734621647252) > weight (0.25) 2021-02-21T00:53:42.229 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6470195341809539) > weight (0.25) 2021-02-21T00:53:42.229 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7435859414240209) > weight (0.25) 2021-02-21T00:53:42.229 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-21T00:54:27.026 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-21T00:54:27.027 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-21T00:54:27.028 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-21T00:54:27.029 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-21T00:54:27.023+0000 7ff38bd3b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 89460) UID: 1000 2021-02-21T00:54:27.229 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:54:29.029 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-21T00:54:29.029 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:54:29.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:54:29.030 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-21T00:54:29.031 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-21T00:54:29.031 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-21T00:54:29.035 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-21T00:54:29.035 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-21T00:54:49.466 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-21T00:54:49.459+0000 7f03ccd38700 -1 reset not still connected to 0x556b752fb790 2021-02-21T00:54:49.467 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-21T00:54:49.459+0000 7f03ccd38700 -1 reset not still connected to 0x556b826008f0 2021-02-21T00:54:49.467 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-21T00:54:49.459+0000 7f03ccd38700 -1 reset not still connected to 0x556b82600a90 2021-02-21T00:54:49.467 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-21T00:54:49.459+0000 7f03ccd38700 -1 reset not still connected to 0x556b826027d0 2021-02-21T00:54:49.468 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-21T00:54:49.459+0000 7f03ccd38700 -1 reset not still connected to 0x556b82602a40 2021-02-21T00:54:49.468 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-21T00:54:49.459+0000 7f03ccd38700 -1 reset not still connected to 0x556b82603330 2021-02-21T00:54:49.468 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-21T00:54:49.459+0000 7f03ccd38700 -1 reset not still connected to 0x556b826039b0 2021-02-21T00:54:49.468 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-21T00:54:49.459+0000 7f03ccd38700 -1 reset not still connected to 0x556b85c86c30 2021-02-21T00:54:49.468 INFO:tasks.ceph.cluster2.osd.1.smithi183.stderr:2021-02-21T00:54:49.459+0000 7f03ccd38700 -1 reset not still connected to 0x556b85c89e90 2021-02-21T00:54:59.204 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-21T00:54:59.194+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a9c704e0 2021-02-21T00:54:59.205 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-21T00:54:59.198+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a9c709c0 2021-02-21T00:54:59.205 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-21T00:54:59.198+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a9c71520 2021-02-21T00:54:59.206 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-21T00:54:59.198+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a9c727d0 2021-02-21T00:54:59.206 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-21T00:54:59.198+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6a9c73260 2021-02-21T00:54:59.206 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-21T00:54:59.198+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6b38350c0 2021-02-21T00:54:59.206 INFO:tasks.ceph.cluster2.osd.2.smithi183.stderr:2021-02-21T00:54:59.198+0000 7fcbd16b5700 -1 reset not still connected to 0x55c6cfb65860 2021-02-21T00:55:39.231 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-21T00:55:39.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-21T00:55:39.232 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-21T00:55:39.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:2021-02-21T00:55:39.225+0000 7f473f406700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 89267) UID: 1000 2021-02-21T00:55:39.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:55:43.232 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-21T00:55:43.233 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:55:43.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:55:43.234 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-21T00:55:43.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-21T00:55:43.234 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-21T00:55:43.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-21T00:55:43.239 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-21T00:55:55.036 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6971534669563961) > weight (0.25) 2021-02-21T00:55:55.036 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8287441696684038) > weight (0.25) 2021-02-21T00:55:55.036 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4321690674514346) > weight (0.25) 2021-02-21T00:55:55.037 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6086992105509977) > weight (0.25) 2021-02-21T00:55:55.037 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-21T00:57:02.037 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-21T00:57:02.039 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-21T00:57:02.039 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-21T00:57:02.040 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-21T00:57:02.035+0000 7f4929fe5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 89552) UID: 1000 2021-02-21T00:57:02.240 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:57:09.040 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-21T00:57:09.040 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:57:09.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:57:09.041 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-21T00:57:09.042 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-21T00:57:09.042 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-21T00:57:09.048 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-21T00:57:09.048 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-21T00:57:13.240 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.30646508863343214) > weight (0.25) 2021-02-21T00:57:13.240 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-21T00:57:13.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-21T00:57:13.241 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-21T00:57:13.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:2021-02-21T00:57:13.239+0000 7f933c6f8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 89085) UID: 1000 2021-02-21T00:57:13.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:57:18.242 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-21T00:57:18.243 INFO:teuthology.orchestra.run:waiting for 600 2021-02-21T00:57:18.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-21T00:57:18.244 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-21T00:57:18.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-21T00:57:18.245 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-21T00:57:18.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-21T00:57:18.247 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-21T00:58:26.049 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-21T00:58:26.058 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-21T00:58:26.058 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-21T00:58:26.059 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:2021-02-21T00:58:26.053+0000 7f1aed805700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 89828) UID: 1000 2021-02-21T00:58:26.260 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi183.stderr:daemon-helper: command failed with exit status 1 2021-02-21T00:58:29.989 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2021-02-21T00:58:29.994 DEBUG:teuthology.task.console_log:Killing console logger for smithi170 2021-02-21T00:58:29.995 DEBUG:teuthology.task.console_log:Killing console logger for smithi183 2021-02-21T00:58:29.995 DEBUG:teuthology.exit:Finished running handlers