# install ansible in virtual environmenty source bin/activate python -m pip install ansible # The below is required so ansible recognises its path (deactivate and activate) deactivate && source bin/activate ansible --version ---- Before we start is important to know the **variable precedence**. See this [[https://subscription.packtpub.com/book/networking_and_servers/9781787125681/1/ch01lvl1sec13/variable-precedence|External Link]] and most important bits below: - Extra vars (from command-line) always win - Play ''vars_files'' - Playbook ''host_vars'' - Playbook ''group_vars'' - Inventory ''host_vars'' - Inventory ''group_vars'' - Inventory ''vars'' Here to find a quick **definition** of //tasks, roles, plays, playbooks// and so on [[https://devops.stackexchange.com/questions/9832/ansible-whats-the-difference-between-task-role-play-and-playbook|External Link]] \\ Modules can be: standard, installed or included in the project itself as [[https://auscunningham.medium.com/write-a-ansible-module-with-python-527f0b292b4d|python-scripts]], we just add a folder ''library'' and we add the modules as ''name_of_the_module.py''. libraries I might require on those py scripts are: ''configparser, git, shutil, AnsibleModule'' ---- cat > ansible.cfg [defaults] gathering = explicit inventory = inv.yml. # calling it hosts can be very confusing, better inv.yml retry_files_enabled = False # disable host checking to automatically add hosts to known_hosts host_key_checking = False **JINJA2 TEMPLATES:**\\ To use it in local (to test things or just to generate configurations to use them later)[[https://github.com/ipspace/NetOpsWorkshop/tree/master/Jinja2/ipaddr|External Link]]: * transport=local # in ansible.cfg * localhost # in the hosts file * hosts: localhost & connection: local # in the main playbook * Training: * {{ :automation:ansible-course-1.pdf |}} * Shorter version: {{ :automation:ansible-main.pdf |}} * {{ :automation:ipspace-jinja2.pdf |}} * [[https://github.com/sandervanvugt/ansiblefundamentals|Sander1]], [[https://github.com/sandervanvugt/ansiblein3weeks|Sander2]] ---- IPADDR FILTERS:\\ They are used extensively in j2 to validate and manipulate IPs. See the official docs and the cheatsheet: * [[https://nwmichl.net/2020/05/25/working-with-ipaddr-in-ansible/|ipaddr-chearsheet]] * [[https://ttl255.com/ansible-ipaddr-filter/]] ---- FLAGS USAGE:\\ ''ansible_run_tags'' reserved keywork: Contents of the --tags CLI option, which specifies which tags will be included for the current run. Note that if --tags is not passed, this variable will default to ["all"].\\ So in theory this can only be used when starting an ansible script via CLI. **However we can write a central playbook and then divert to other subplaybooks depending on the tag passed by any shceduling system like AWX**: name: Process Port-Channel Summary on Cisco NXOS import_playbook: port_channel/playbook_nxos.yml when: - "'nxos' in ansible_run_tags" - "'port_channel' in ansible_run_tags" name: Process Port-Channel Summary on Cisco IOSXE import_playbook: port_channel/playbook_iosxe.yml when: - "'iosxe' in ansible_run_tags" - "'port_channel' in ansible_run_tags" Tags can also be set inside the play itself and inherited. See this [[https://www.percona.com/blog/2020/04/27/how-do-ansible-tags-work/|External Link]] ---- AD-HOC COMMANDS:\\ 'raw' command > (unlike shell) raw does not require python to be installed in the target server, hence so used with network devices * -m : module * -a : arguments ansible all -i inventory -m setup # this collects facts ansible -i hosts all -m ping --private-key /home/student/ansible_key # PING module, useful to test connectivity to the hosts ansible all -i "sw-b05.dc.mycompany1.co.uk," -m raw -a "show version" -u jaime_santos # comma and double quotes are needed ansible -i ini/hosts "linux1," -m raw -a "/usr/sbin/ip route" ansible all -m raw -a "show version" -u jaime_santos -i ini/hosts # raw module doesn't require python installed in the remote 'router' ansible -m command -a "grep lisa /etc/shadow" # this is more for linux, it uses module 'command' The 'command' module above ^, when used in the playbook, dumps the output in the 'register' variable \\ PLAYBOOKS ---- __YAML SYNTAX:__\\ Follow this recommendatrions: [[https://learn.redhat.com/t5/Automation-Management-Ansible/Ansible-Tips-and-Tricks/td-p/148|External Link]] * Call all playbooks playbook.yml and keep in separate folders * Add this to vimr * //autocmd FileType yaml setlocal ai ts=2 sw=2 et// ansible-playbook -i vyos.example.net, -u jaime_santos first_playbook.yml ansible-playbook -i ini/hosts -l all get_dev_facts_screen.yml # Output in screen, facts defined in the playbook ansible-playbook -i ini/hosts -l all get_all_facts.yml # Output in 'results_facts' folder, all available facts downloaded. FLAGS: * -i inventory * -u connect as this User * -e set additional variables: example -e ansible_network_os=vyos * -l to select a group or an individual server from ini/hosts file For more **info**: [[https://docs.ansible.com/ansible/2.4/ansible-playbook.html|External Link]] \\ ---- ANSIBLE.CFG NOTES: * we can place an ansible.cfg file in our project folder. It will override the default one * **host_key_checking = false** * gather_facts: no # this generally goes in the play level \\ ---- Note example below where we place all variable in a var file: ~ hosts: crpnycnetdev02 connection: local gather_facts: no vars_files: ~ vars/sflinx.yml ~ vars/env_site.yml tasks: ~ name: Generate Configsc template: src: "templates/cspolicy.j2" dest: "configs/api_{{filter_condition}}_{{item.key}}.cfg" with_dict: "{{ sites }} The var file simply looks like this: ~~~ # ./vars/name_vars.yml name: World # then, in another playbook, the var can be referred as {{name}} This is **another** way to parse jinja2 templates into result files [[https://docs.ansible.com/ansible/latest/collections/ansible/builtin/template_lookup.html|External Link]]. Remember **lookups are just plugins**, so multiple functions. See [[https://docs.ansible.com/ansible/latest/plugins/lookup.html#plugin-list|External Link]] set_fact: data="{{lookup('template','templates/node-model.j2')}}" # returns a string containing the results of processing that template. ---- **__ACCESS TO DICTIONARY EXAMPLES AND MORE__**:\\ Example of lists and dictionaries. If we add **register** to the play's loop, the variable specified will contain all responses from the module: ~~~ ~ name: Test loop over list and dict hosts: all become: true vars: my_grocery_list: ~ milk ~ butter my_car_preferences: brand: mercedes model: G tasks: ~ name: Loop over list debug: msg: "(( item }}" loop: "{{ my_grocery_list }}" tasks: ~ name: Loop over dict debug: msg: "{{ item.key }} {{ item.value }}" loop: "{{ my_car_preferences | dict2items }}" __ITERITEMS AND MAP__ If we want a loop on a dictionary, we use ''dict2items'' !! This is the template ! note: 'map' filter does not transform. It either applies a filter to all a sequence of objects and/or looks up an attribute. { "vlans": { {% for customer,svc in services.iteritems() if inventory_hostname in svc.ports|map(attribute='node') and svc.state|default("") != "absent" %} "{{svc.vlan}}": "{{customer}}"{% if not(loop.last) %},{% endif %} {% endfor %} ! !! This is the vars file (included in the play with ''include_vars'') --- ACME: vlan: 100 ports: - { node: leaf-1, port: "Ethernet2/1", site: "ACME Downtown" } - { node: leaf-2, port: "Ethernet2/3", site: "ACME Remote" } Wonka: ... >>> with_dict: "{{ sites }} >>> loop: "{{ my_car_preferences | dict2items }}" >>> if inventory_hostname in svc.ports|map(attribute='node') ---- \\ # Configuration management. After templeting, we need to push the configurations to the box - opt 1: built in modules. - opt1.1: napalm - ntc # multi-vendor based on netmiko. eg: ntc_show_command # ansible-playbook playbook.yml --tags-push # ansible-playbook playbook.yml --tags-push --limit csr1 # limit this job to this group/device # ansible-playbook playbook.yml --tags=connfig **Example of a module's parameters(1) in playbooks:**\\ (1) (I would call them commands instead). See [[https://docs.ansible.com/ansible/asa_config_module.html]] \\ * match * lines * provider ---- INVENTORY\\ ansible-inventory -i ./hosts.yml --list # to expand and verify the inventory file We can also use **dynamic inventories** (mostly useful when we connect to clouds). There are py scripts for this which , for example spits in json format all hosts it finds in dns etc/hosts and anywhere else. ---- VARIABLES:\\ These are all the [[https://docs.ansible.com/ansible/latest/reference_appendices/special_variables.html|Reserved Variables in Ansible]] * playbook_dir : is a tring with the dictory where the playbook is running. * inventory_hostname : the ‘current’ host being iterated over in the play * example: dest: {{ playbook_dir }}/output/{{ inventory_hostname }}_facts.txt \\ * If we want to check the value of variables, before doing anything we them, we can use the //module debug// * Useful when dealing with managed hosts where specifics are different. * We use the **vars** (note the plural) module in the playbooks Variable: referenced with Double Curly braces {{var}} \\ Use subl to edit/create the yaml files. See if pycharm is up for it too or just np++ can equally do. Variables can be: * In the own playbook * var field ~ hosts: all vars: web_package: httpd * Somewhere else: * inventory file ( these are possible variables in the inventory [[https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html#connecting-to-hosts-behavioral-inventory-parameters|Link]] ) * **From a file** ( vars_files or include_vars ) : vars_files are read when the play starts. include_vars are read when the play reaches the task. You probably might be interested also in Variable precedence. More info [[https://stackoverflow.com/questions/53253879/ansible-vars-files-vs-include-vars|here]] ~ hosts: all vars_files: ~ var/users.yml * inventory vars * var * var_file list * 'magic' variables: //hostvars, groups, group_names, inventory_hostname// check this [[https://docs.ansible.com/ansible/latest/user_guide/playbooks_variables.html#accessing-information-about-other-hosts-with-magic-variables|External Link]] * IMPORTANT VARIABLES: * [[https://github.com/ansible/ansible/blob/devel/docs/docsite/rst/network/user_guide/platform_index.rst|ansible_network_os]] : can be: ''ios'' (classical and ios xe), ''iosxr'', ''nxos'' .. * ansible_user * ansible_password {{:automation:ansiblevar11.jpg?200|}} Precedence can be found in this [[http://docs.ansible.com/ansible/playbooks_variables.html#variable-precedence-where-should-i-put-a-variable|Link]]. \\ Basically is : 1.- extra vars (-e in the command line) always win ; 2.- vars in play ; 3.- play vars_files ; 4.- facts ; 5.- inventory \\ http://docs.ansible.com/ansible/playbooks_variables.html # - inventory_hostname is the name of the hostname as configured in Ansible’s inventory host file. ansible-playbook -e "var=value" # overrides everything ---- REGEX:\\ To modify parts of existing configuration files depending on what is already there: **lineinfile**. It uses classical linux regex types. ~ name: Ensure SELinux is set to enforcing mode lineinfile: path: /etc/selinux/config regexp: '^SELINUX=' line: SELINUX=enforcing ---- CONDITIONALS:\\ * **handlers** * notify the event the handler listens to, to be executed. then we have handler keyword instead of 'task'. The name of the notify and the handler is the same. Is the link between them two * handlers have a **main.yml**. It defines handlers used in the role * **items** (iterations) * **when**. (conditional) * **blocks** ( if-then-else ) * **register** ( to store the result of a task in a variable ) * **fail** ( to catch failures ) ---- LOOPS TO PROCESS A LIST OF ITEMS:\\ * Use ''loop'' (legacy: or with_) * it uses ''item'' keyword to go through the list. \\ ~~~ - name: install and start services hosts: ansible1 tasks: - name: install packages yum: name: - vsftpd - httpd - samba state: latest - name: start the services service: name: "{{ item }}" state: started enabled: yes loop: - vsftpd - httpd - smb * This is a loop traversing a **list of dictionaries**: [[https://github.com/sandervanvugt/ansiblefundamentals/blob/master/loop_on_vars.yaml]] * Legacy with_ syntax * with_items is equivalent to modern ''loop'' * ''with_file'' : the item contains a file, which contents are used to loop through * In modern ansible, most of the ''with'' can be replaced with **filters** ---- FACT GATHERING:\\ - All via the setup module - It goes to a **variable called "{{ ansible_facts }} "**which is not an env variable and is only accessible via ansible. **Nonetheless we can dump them to a file**. ansible all -m setup -a "filter=*ipv4" ansible ubuntu -m setup # adhoc command, to see the whole fact ansible -i host localhost -m setup This is how you go down the facts json tree and select one field. This to add the different parts of the fact (list, dictionaries..) [[https://www.middlewareinventory.com/blog/ansible-facts-list-how-to-use-ansible-facts/|Link]] ~~~ ~ name: Facts2 hosts: localhost gather_facts: yes tasks: ~ name: task1 debug: msg: "{{ ansible_facts['distribution'] }}" This is to apply a when statement to the result of a fact section: ~~~ ~ name: Facts2 hosts: localhost tasks: ~ name: show fact element debug: msg: "{{ ansible_default_ipv4.mtu }}" ~ name: do something when mtu is 9000 debug: msg: "THE_INTERFACE_IS_JUMBO!!!" when: ansible_default_ipv4.mtu == 9000 This is how you go down the facts json tree and assign the value to a variable: TODO ~~TODO~~ ---- __ASSERT__ \\ Checks the **veracity** of an expression A bot like the fail module but with more advanced features. * Also uses these keywords: ''fail_msg'' and ''success_msg'' \\ Example: ~~~ - hosts: localhost vars_prompt: - name: filesize prompt: "specify file size" private: no tasks: - name: check file size is valid assert: that: - filesize <= 100 - filesize >= 1 fail_msg: "size has to be between 0 and 100!" success_msg: "correct size, continue" ---- **NOW FOR JUNOS DEVICES**\\ * Core junos modules included with ansible: [[https://docs.ansible.com/ansible/latest/modules/list_of_network_modules.html#junos|Link]] * Be sure this is in the ansible conf or you will get //"msg": "name 'known_hosts_lookup' is not defined"// * host_key_checking = False * There's no python running in the remote device (switch), so you need this in the playbook!: * connection: local * This is to verify netconf is working in the net device: ---- __**ANSIBLE DEGUB (ansible troubleshooting)**__: \\ Check this guide/section: [[https://docs.ansible.com/ansible/latest/network/user_guide/network_debug_troubleshooting.html#enabling-networking-logging-and-how-to-read-the-logfile|docs.ansible.com]] # Specify the location for the log file export ANSIBLE_LOG_PATH=~/ansible.log # Enable Debug export ANSIBLE_DEBUG=True # Run with 4*v for connection level verbosity ansible-playbook -vvvv ... ---- __**ROLES**__ * Roles can be downloaded from galaxy as part of collections. * We can also ''create our own roles'': * first we create the dir with subfolders: defaults, handlers, meta, tasks, templates, vars * This is an example of a custom role: [[https://github.com/sandervanvugt/rhce8-live/tree/0fd56c840f683af72a74647ad2f7118555153425/lesson11/roles/motd]] ansible-galaxy role init ---- __roles for Junos __ \\ * Roles (network engine role) act as wrappers. They allows us to use a yaml based parser (command_parser) instead of custom filters. This way, we make the code much shorter. * ansible-galaxy is a collection of roles * ~In the device: set system services netconf traceoptions [file netconf , lag all ] * From a linux box: ssh jaime_santos@loncr02.dc -t netconf # to exit the session * This is to enable full local logs for ansible **The below is part of the junos_facts (galaxy juniper.junos module) [[https://galaxy.ansible.com/Juniper/junos|Juniper.junos]]. Use these as examples:**[[https://junos-ansible-modules.readthedocs.io/en/2.4.0/]] ; Junos galaxy page: [[https://galaxy.ansible.com/Juniper/junos|Link]] ~~~ ~ name: Facts2 hosts: GYRON connection: local gather_facts: no roles: ~ Juniper.junos tasks: ~ name: Gather Junos facts with no configuration juniper_junos_facts: Example of [[https://panda314159.duckdns.org/doku.php?id=automation:ansible:junos_facts-output1|juniper_junos_facts]] ---- __ENABLE SSH KEY AUTHENTICATION__ \\ ---- **__MY TEMPLATES__**\\ {{ :automation:add_addresses_asa1.txt |}} \\ {{ :automation:ansible_config_generator1.txt |}} ---- ---- **ACUMOS PROJECT (linux fundation project, machine learning for networking):**\\ [[https://www.acumos.org/author/acumos/]] [[https://www.linuxfoundation.org/press-release/2018/03/the-linux-foundation-launches-open-source-acumos-ai-project/]] ---- All tools in a docker instance:: [[https://packetpushers.net/building-a-docker-network-automation-container/]] ---- __ANSIBLE CUSTOM FILTERS__ \\ See Pluralsight folder: ''PLURALSIGHT/automating-networks-ansible-right-way/05/demos/m5/tests/tasks/test_rt_diff.yml'' \\ # Note that for a filter to get two arguments, first is left to the pipe and rests are between parenthesis - name: "Find route-target differences" set_fact: rt_updates: "{{ int_vrf_list | rt_diff(run_vrf_dict) }}" ---- __MANAGE FAILURES__ \\ * ''ignore_errors'' * ''force_handlers'' : to force a handler that has been triggered to run, even if (another) task fails. * [[https://stackoverflow.com/questions/42653655/ansible-ignore-errors-when|External Link]] ---- __AWX__ \\ * Ansible Tower == AWS =~ 'Ansible Automation Platform' (this latter is sold now by redhat) * {{ :automation:awx-sander-van-vugt.pdf |}} * **AWX uses k8s extensively** Install minikube (ubuntu) with: minikube start --cpus=4 --memory=6g --addons=ingress --vm-driver=docker