Ansible最佳实践

Ansible Best Practices

Posted by BlueFat on Wednesday, December 12, 2018

ansible https://docs.ansible.com/
Ansible优化 https://ansible.leops.cn/advanced/optimization/

Ansible Best Practices

Ansible Best Practices

inventories/
   production/
      hosts               # inventory file for production servers
      group_vars/
         group1.yml       # here we assign variables to particular groups
         group2.yml
      host_vars/
         hostname1.yml    # here we assign variables to particular systems
         hostname2.yml

   staging/
      hosts               # inventory file for staging environment
      group_vars/
         group1.yml       # here we assign variables to particular groups
         group2.yml
      host_vars/
         stagehost1.yml   # here we assign variables to particular systems
         stagehost2.yml

library/                  # if any custom modules, put them here (optional)
module_utils/             # if any custom module_utils to support modules, put them here (optional)
filter_plugins/           # if any custom filter plugins, put them here (optional)

site.yml                  # master playbook
webservers.yml            # playbook for webserver tier
dbservers.yml             # playbook for dbserver tier

files/                    # here we assign files for simple plays
plays/                    # here we assign plays as the entrance
tasks/                    # here we assign tasks for plays to call

roles/
    common/               # this hierarchy represents a "role"
        tasks/            #
            main.yml      #  <-- tasks file can include smaller files if warranted
        handlers/         #
            main.yml      #  <-- handlers file
        templates/        #  <-- files for use with the template resource
            ntp.conf.j2   #  <------- templates end in .j2
        files/            #
            bar.txt       #  <-- files for use with the copy resource
            foo.sh        #  <-- script files for use with the script resource
        vars/             #
            main.yml      #  <-- variables associated with this role
        defaults/         #
            main.yml      #  <-- default lower priority variables for this role
        meta/             #
            main.yml      #  <-- role dependencies
        library/          # roles can also include custom modules
        module_utils/     # roles can also include custom module_utils
        lookup_plugins/   # or other types of plugins, like lookup in this case

    webtier/              # same kind of structure as "common" was above, done for the webtier role
    monitoring/           # ""
    fooapp/               # ""

提升 Ansible 执行效率的插件

众所周知,Ansible 是基于 ssh(当然还有 telnet,winrm 等连接插件)的自动化配置管理工具,其简单易用,无 agent 式的工作方式在很多场景中都有不少优势,不过也是由于这种工作方式导致了它没有其他 c/s 类的工具执行效率高,饱受其他 C/S 类工具使用者的讥讽,对此,Ansible 官方也对 Ansible 的速度效率做了不少优化手段。

参数名 / 优化类别 说明
fact cache 将 facts 信息第一次收集后缓存到 memory 或者 redis 或者文件中。
gather_subset 可选择性的收集 network,hardware 等信息,而不是全部
control_path 开启 ssh socket 持久化,复用 ssh 连接
pipelinling 开启 ssh pipelining, 客户端从管道中读取执行渲染后的脚本,而不是在客户端创建临时文件
fork 提高并行执行主机的数量
serial play_hosts``① 中主机再分批执行
strategy 默认 linear, 每个主机的单个 task 执行完成会等待其他都完成后再执行下个任务,设置 free 可不等待其他主机,继续往下执行(看起来会比较乱),还有一个选项 host_pinned,我也不知道干嘛的

无意发现了一个 Mitogen 的 Ansible plugin(strategy plugin),当前已迭代到 0.29 版本(目前只支持2.9版本),看介绍说能提升 1.2x ~ 7x 以上的执行效率,着实惊人!

大体就是执行过程中主机使用一个连接(默认每执行一个 task 或者 loop 循环都会重新打开一次连接的);渲染的执行代码暂存于内存中;减少多路复用 ssh 隧道的时间消耗;减少临时文件传输的带宽;代码重用,避免代码的重新编译成本等

①. play_hosts 为内置参数,指当前正在执行的 playbook 中的主机列表

②. 尽可能快的 指到运行模块前的阶段h Download and extract mitogen-0.2.9.tar.gz Modify ansible.cfg

[defaults]
strategy_plugins = /path/to/mitogen-0.2.9/ansible_mitogen/plugins/strategy
strategy = mitogen_linear

https://networkgenomics.com/ansible/ https://mitogen.networkgenomics.com/ansible_detailed.html

ansible.cfg 配置解析

| ansible.cfg 不影响执行结果但合理的配置会有效提升效率

# 配置文件路径(优先级)
./ansible.cfg
/etc/ansible/ansible.cfg

# 配置文件内容
[defaults]
#inventory = /etc/ansible/hosts
#log_path = /var/log/ansible.log
forks = 100 # 设置并发数
host_key_checking = False # 不检查 SSH 主机登录的密钥
display_skipped_hosts = False # 不显示已跳过的主机
retry_files_enabled = False # 不创建任务失败后的重试文件
# 按照 1d 设置 setup 缓存,优化执行效率
gathering = smart
fact_caching_timeout = 86400
fact_caching = jsonfile
fact_caching_connection = /tmp
roles_path = ./roles

[ssh_connection]
pipelining = True
ssh_args = -o ConnectionAttempts=100 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ForwardAgent=yes

[inventory]
enable_plugins = yaml, ini

核心用法

Linux

# 检测 ansible 是否可以正常访问主机
ansible-playbook -i hosts playbooks/ping.yml -v
# 配置好 inventory,执行以下命令创建用户并建立信任关系
ansible-playbook -i hosts playbooks/user/default.yml -v
# 配置时间同步 / 进程服务 / 基线文件
ansible-playbook -i hosts playbooks/baseline/cfgset.yml -v
ansible-playbook -i hosts playbooks/baseline/cfgset.yml -v --tags="repo"
ansible-playbook -i hosts playbooks/baseline/cfgset.yml -v --skip-tags="ntp,repo"
# 更新系统软件包和补丁包
ansible-playbook -i hosts playbooks/baseline/pakset.yml -v
# 修改用户密码
ansible-playbook -i hosts_changepw playbooks/user/changepw.yml -v -e "@userpass.json"
# 备份配置,支持自定义日期命名,默认为 "%Y%m%d"
ansible-playbook -i hosts backup/backup.yml -v
# 恢复配置,支持按日期目录全局或者局部主机恢复
ansible-playbook -i hosts backup/restore.yml -v -e "var_backup_date=20180305"

Windows

# 检测 ansible 是否可以正常访问主机
ansible-playbook -i hosts win_playbooks/ping.yml -v
# 配置好 inventory,执行以下命令创建用户并建立信任关系
ansible-playbook -i hosts win_playbooks/user/default.yml -v
# 配置时间同步 / 进程服务 / 基线文件
ansible-playbook -i hosts win_playbooks/baseline/cfgset.yml -v
ansible-playbook -i hosts win_playbooks/baseline/cfgset.yml -v --tags="wsus"
ansible-playbook -i hosts win_playbooks/baseline/cfgset.yml -v --skip-tags="ntp,wsus"
# 更新系统软件包和补丁包
ansible-playbook -i hosts win_playbooks/baseline/pakset.yml -v
# 修改用户密码
ansible-playbook -i win_hosts_changepw win_playbooks/user/changepw.yml -v -e "@userpass.json"
# 备份配置,支持自定义日期命名,默认为 "%Y%m%d"
ansible-playbook -i win_hosts win_backup/backup.yml -v
# 恢复配置,支持按日期目录全局或者局部主机恢复
ansible-playbook -i win_hosts win_backup/restore.yml -v -e "var_backup_date=20180305"

Where X=ansible

https://learnxinyminutes.com/docs/ansible/

---
"{{ Ansible }}" is an orchestration tool written in Python.
...

---
- hosts: apache

  vars:
      apache2_log_level: "warn"

  handlers:
  - name: restart apache
    service:
      name: apache2
      state: restarted
      enabled: True
    notify:
      - Wait for instances to listen on port 80
    become: True

  - name: reload apache
    service:
      name: apache2
      state: reloaded
    notify:
      - Wait for instances to listen on port 80
    become: True

  - name: Wait for instances to listen on port 80
    wait_for:
      state: started
      host: localhost
      port: 80
      timeout: 15
      delay: 5

  tasks:
  - name: Update cache
    apt:
      update_cache: yes
      cache_valid_time: 7200
    become: True

  - name: Install packages
    apt:
      name={{ item }}
    with_items:
      - apache2
      - logrotate
    notify:
      - restart apache
    become: True

  - name: Configure apache2 log level
    lineinfile:
      dest: /etc/apache2/apache2.conf
      line: "LogLevel {{ apache2_log_level }}"
      regexp: "^LogLevel"
    notify:
      - reload apache
    become: True
...

# Universal way
$ pip install ansible

# Debian, Ubuntu
$ apt-get install ansible

# Command pings localhost (defined in default inventory: /etc/ansible/hosts)
$ ansible -m ping localhost
# You should see this output
localhost | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

$ ansible -m ping all
$ ansible -m shell -a 'date; whoami' localhost #hostname_or_a_group_name

$ ansible -m command -a 'date; whoami' # FAILURE
$ ansible -m command -a 'date' all
$ ansible -m command -a 'whoami' all

- hosts: all

  tasks:
    - name: "ping all"
      ping:

    - name: "execute a shell command"
      shell: "date; whoami; df -h;"

$ ansible-playbook path/name_of_the_playbook.yml

localhost

[some_group]
hostA.mydomain.com
hostB.localdomain
1.2.3.4

[a_group_of_a_groups:children]
some_group
some_other_group

- hosts: all

  tasks:
      - name: "ping all"
        ping:
      - name: "execute a shell command"
        shell: "date; whoami; df -h;"

  roles:
      - some_role
      - { role: another_role, some_variable: 'learnxiny', tags: ['my_tag'] }

  pre_tasks:
      - name: some pre-task
        shell: echo 'this task is the last, but would be executed before roles, and before tasks'

$ # The following example contains a shell-prompt to indicate the venv and relative path
$ git clone git@github.com:sirkubax/ansible-for-learnXinYminutes.git
user@host:~/$ cd ansible-for-learnXinYminutes
user@host:~/ansible-for-learnXinYminutes$ source environment.sh
$
$ # First lets execute the simple_playbook.yml
(venv) user@host:~/ansible-for-learnXinYminutes$ ansible-playbook playbooks/simple_playbook.yml

$ source environment.sh
$ # Now we would run the above playbook with roles
(venv) user@host:~/ansible-for-learnXinYminutes$ ansible-playbook playbooks/simple_role.yml

roles/
   some_role/
     defaults/      # contains default variables
     files/         # for static files
     templates/     # for jinja templates
     tasks/         # tasks
     handlers/      # handlers
     vars/          # more variables (higher priority)
     meta/          # meta - package (role) info

playbooks/roles/simple_apache_role/
├── tasks
│   └── main.yml
└── templates
    └── main.yml

# read playbooks/lookup.yml
# then run
(venv) user@host:~/ansible-for-learnXinYminutes$ ansible-playbook playbooks/lookup.yml

ansible -m shell -a 'echo "{{ my_variable }}"' -e 'my_variable="{{ lookup("pipe", "date") }}"' localhost
ansible -m shell -a 'echo "{{ my_variable }}"' -e 'my_variable="{{ lookup("pipe", "hostname") }}"' all

# Or use in playbook

(venv) user@host:~/ansible-for-learnXinYminutes$ ansible-playbook playbooks/lookup.yml

(venv) user@host:~/ansible-for-learnXinYminutes$ ansible-playbook playbooks/register_and_when.yml

---
- hosts: localhost
  tasks:
   - name: check the system capacity
     shell: df -h /
     register: root_size

   - name: debug root_size
     debug:
        msg: "{{ root_size }}"

   - name: debug root_size return code
     debug:
       msg:  "{{ root_size.rc }}"

# when: example

   - name: Print this message when return code of 'check the system capacity' was ok
     debug:
       msg:  "{{ root_size.rc }}"
     when: root_size.rc == 0
...

---
- hosts: localhost
  tasks:
   - name: check the system capacity
     shell: df -h /
     when: some_variable in 'a string'
  roles:
   - { role: mid_nagios_probe, when: allow_nagios_probes }
...

ansible-playbook playbooks/simple_playbook.yml --tags=tagA,tag_other
ansible-playbook playbooks/simple_playbook.yml -t tagA,tag_other

There are special tags:
    always

--skip-tags can be used to exclude a block of code
--list-tags to list available tags

ansible-playbook playbooks/simple_playbook.yml --limit localhost

--limit my_hostname
--limit groupname
--limit some_prefix*
--limit hostname:group #JM

Some static content

{{ a_variable }}

{% for item in loop_items %}
    this line item is {{ item }}
{% endfor %}

$ source environment.sh
$ # Now we would run the above playbook with roles
(venv) user@host:~/ansible-for-learnXinYminutes$ ansible-playbook playbooks/simple_role.yml --tags apache2

ansible -m shell -a 'echo {{ my_variable }}' -e 'my_variable=something, playbook_parameter=twentytwo' localhost

# check part of this playbook: playbooks/roles/sys_debug/tasks/debug_time.yml
- local_action: shell date +'%F %T'
  register: ts
  become: False
  changed_when: False

- name: Timestamp
  debug: msg="{{ ts.stdout }}"
  when: ts is defined and ts.stdout is defined
  become: False

# get first item of the list
{{ some_list | first() }}
# if variable is undefined - use default value
{{ some_variable | default('default_value') }}

# Try (this would fail)
$ ansible-playbook playbooks/vault_example.yml

$ echo some_very_very_long_secret > ~/.ssh/secure_located_file

# in ansible.cfg set the path to your secret file
$ vi ansible.cfg
  ansible_vault_password_file = ~/.ssh/secure_located_file

#or use env
$ export ANSIBLE_VAULT_PASSWORD_FILE=~/.ssh/secure_located_file

$ ansible-playbook playbooks/vault_example.yml

  # encrypt the file
$ ansible-vault encrypt path/somefile

  # view the file
$ ansible-vault view path/somefile

  # check the file content:
$ cat path/somefile

  # decrypt the file
$ ansible-vault decrypt path/somefile

$ etc/inv/ec2.py --refresh
$ ansible -m ping all -i etc/inv/ec2.py

vi ansible.cfg
# set this to:
callback_whitelist = profile_tasks

vi ansible.cfg

# if set to a persistent type (not 'memory', for example 'redis') fact values
# from previous runs in Ansible will be stored.  This may be useful when
# wanting to use, for example, IP information from one group of servers
# without having to talk to them in the same playbook run to get their
# current IP information.
fact_caching = jsonfile
fact_caching_connection = ~/facts_cache
fact_caching_timeout = 86400

# recreate ansible 2.x venv
$ rm -rf venv2
$ source environment2.sh

# execute playbook
(venv2)$ ansible-playbook playbooks/ansible1.9_playbook.yml # would fail - deprecated syntax

# now lets install ansible 1.9.x next to ansible 2.x
(venv2)$ deactivate
$ source environment.1.9.sh

# execute playbook
(venv1.9)$ ansible-playbook playbooks/ansible1.9_playbook.yml # works!

# please note that you have both venv1.9 and venv2 present - you need to (de)activate one - that is all

- name: Ensure the httpd service is running
  service:
    name: httpd
    state: started
  become: true

ansible -m ping web*

ansible -m ping web*:!backend:monitoring:&allow_change

参考

Ansible 学习路径 Ansible Documentation
ansible-workshops
Ansible入门
Ansible Wiki