This is an old revision of the document!
BASIC IP OPERATIONS Assign static IP and gateway (not permanent)
ip address add 10.0.0.3/24 dev eth0 ip addr add 192.168.12.1/24 dev lo # this is for a loopback address
CREATE NEW INTERFACE (permanent):
Annotate name and hw address
ip link show
Generate UUID:
uuidgen <DEVICE>
[jotasan@localhost network-scripts]$ cat ifcfg-eno33554952 NAME=eno33554952 BOOTPROTO=static TYPE=Ethernet ONBOOT=yes MTU=1500 NETMASK=255.255.255.0 GATEWAY=192.168.233.2 IPADDR=192.168.223.133 DEFROUTE=yes PEERDNS=yes PEERROUTES=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_FAILURE_FATAL=no DEVICE=eno33554952
HWADDR=00:0C:29:5D:91:7B UUID=e2faf099-3de3-43d6-aff7-aa0849125343
This a a blank template, case we need the interface to do noting:
DEVICE=enp13s0 BOOTPROTO=none TYPE=Ethernet ONBOOT=yes NAME=enp13s0 UUID="d6068c43-2b0c-47b1-bf59-291a5fccae8e" # use 'uuidgen enp13s0' to generate it
ETHTOOL:
Show:
ethtool em1 # show settings ethtool -t em1 # test hw ethtool -S em2 | egrep -i "drop|buffer|miss|discard|error|collision" # https://www.linuxjournal.com/content/fun-ethtool ethtool -i em1 # driver ethtool -a eth0 # Display just autonegotiation ethtool --change enp0s3 speed 100 duplex full autoneg off ethtool -p eth0 # to make that port BLINK!
NETWORK PERFORMANCE / INTERFACE THROUGHPUT
sar -n DEV 1 3 iftop iostat
NETWORK INTERFACES AND MODULES To find the driver running a port:
ethtool -i eno1 | grep driver driver: i40e
Many mycompany2 10G and 40g use this External Link , more info here
modinfo bnxt_en filename: /lib/modules/4.1.12-94.el7uek.x86_64/updates/bnxt_en.ko version: 1.7.30 description: Broadcom BCM573xx network driver
mycompany2 tool:
/opt/fma/fm/sbin/fmadm
IPV6 IN LINUX
ipv6 hardening, tuning slaac and more: https://linux-audit.com/linux-security-guide-for-hardening-ipv6/
To ping ipv6 (don't forget the outbound interface):
ping6 -I wlp3s0 fe80::ceaf:78ff:fe08:92cc
To see the neighbors:
ip -6 neigh show
Output for 'ip address':
- Network mananger ipv6 privacy (
To create an (ephimeral) static route:
GATEWAY entry in ifcfg-eth
ip route add 192.168.55.0/24 via 192.168.1.254 dev eth1
To create a permanent static route
Open /etc/sysconfig/network-scripts/route-eth0:
vi /etc/sysconfig/network-scripts/route-eth0\\
Append following line:
10.0.0.0/8 via 10.9.38.65 dev eth0
ARP
arp -a # to show the full table. -d to delete an entry ip -s -s neigh flush all # flushes the whole arp table
DHCP:
- DHCP messages:
- DHCPDiscover: src:0.0.0.0 ; dst: broadcast « PACKET HAS BOOTP FLAG UNSET. This tells the server, do not use bcast in your responses
- DHCPOffer : src: server IP ; dst: client-mac + tentative IP Address
- DHCPRequest: src:0.0.0.0 ; dst: broadcast « PACKET HAS BOOTP FLAG UNSET
- DHCPAck: : src: server IP ; dst: client-mac + tentative IP Address
802.1q IN LINUX
ALWAYS double check NetworkManager is NOT running
Check dot1q kernel module is loaded:
modinfo 8021q [root@titan02.dc.mycompany1.co.uk network-scripts]# lsmod | grep 8021q 8021q 33159 0 garp 14384 1 8021q mrp 18542 1 8021q
Configure the interfaces:
/etc/sysconfig/network-scripts/ifcfg-ethX DEVICE=ethX TYPE=Ethernet BOOTPROTO=none ONBOOT=yes
TAGGING EXAMPLE
NAME=bond0.400 DEVICE=bond0.400 IPV6INIT=no BOOTPROTO=none ONBOOT=yes MTU=9000 NETMASK=255.255.254.0 IPADDR=10.8.14.75 VLAN=yes TYPE=Vlan
BONDING EXAMPLE 802.3ad (LAG) Dynamic link aggregation with LACP
ALWAYS double check NetworkManager is NOT running
IPV6INIT=no NAME=bond0 BONDING_MASTER=yes BOOTPROTO=none BONDING_OPTS="mode=802.3ad miimon=100 lacp_rate=fast xmit_hash_policy=layer2+3" # air quotes are important here! DEVICE=bond0 ONBOOT=yes
Runtime commands:
ip link add bond0 type bond ip link set bond0 type bond miimon 100 mode active-backup ip link set eth1 down ip link set eth1 master bond0 ip link set eth2 down ip link set eth2 master bond0 ip link set bond0 up ip address add 10.0.192.2 dev bond0
LACP NOTES: Is a control protocol on top of the LAG, it prevents things like blackholing traffic if we configure one of the members' end incorrectly or it mantains keepalives to check link health (for example, without lacp, if we had a media converter, a section of the logical link might be down while the other side wouldn't notice and would keep sending traffic).
Also x2 check that lacp fast/slow matches in both sides:
/etc/sysconfig/network-scripts/ifcfg-eth0.192
External Link Review hashing algoritms (xmit_hash_policy) here: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/networking_guide/sec-using_channel_bonding
Most of the LACP theory and troibleshooting (in routers) can be found in this article: http://networkingbodges.blogspot.com/2012/12/all-sorts-of-things-about-lacp-and-lags.html
- System ID(chassis-id): This is a 48 bit value which generally defaults to the chassis MAC address. The system ID is sent within every LACPDU and makes it easy to check that a LAG goes to the device you expect.
- key (mc-ae-id): 16 bit LACP key, the purpose of which is to differentiate one LAG from another within the protocol. The main purpose of the LACP key is to allow a system to detect cabling faults - if different LACP keys are received on members of the same LAG then we are connected to two different LAGs at the far end and, obviously, aggregating those together would be a bad idea.
- port ID: (or port priority. The higher the priority value the lower the priority. Range is 1 to 65535. LACP uses the port priority in combination with the port number to form the port identifier. The port priority is used with the port number to form the port identifier. The port priority is used to decide which ports should be put into standby mode when there is a hardware limitation that prevents all compatible ports from aggregating.
- admin-key
BONDING NOTES
Consider a more modern solution, in user space, like TEAMING https://www.redhat.com/en/blog/if-you-bonding-you-will-love-teaming
Link
url documentation HERE: https://www.kernel.org/doc/Documentation/networking/bonding.txt
http://www.crucial.com.au/blog/2012/11/01/linux-bonding-on-centos-5/
https://wiki.linuxfoundation.org/networking/bonding
cat /proc/net/bonding/bond0 # to see the bond members cat /sys/class/net/bond-lan/bonding/slaves # to see the slaves
- mode=balance-alb : Adaptive load balancing: includes balance-tlb plus receive load balancing (rlb) for IPV4 traffic, and does not require any special switch support. The receive load balancing is achieved by ARP negotiation. The bonding driver intercepts the ARP Replies sent by the local system on their way out and overwrites the source hardware address with the unique hardware address of one of the slaves in the bond such that different peers use different hardware addresses for the server.
- miimon=100 : Use integer value for the frequency (in ms) of MII link monitoring. The MII monitor monitors only the carrier state of the local network interface.
TROUBLESHOOTING:
Lacp not starting the server:
- Check the air quotes in BONDING_OPTS (can work both ways)
- BONDING_OPTS=mode=4 miimon=100 lacp_rate=1
- BONDING_OPTS=“mode=4 miimon=100 lacp_rate=1”
- Check if the lacp and tagging module are loaded. unload and load it
lsmod | grep bonding lsmod | grep 8021q
lsmod: status of the module ; modprobe unloads it ; modprobe -r loads it
TEST JUMBO FRAMES 9000
ping -c 4 -s 8972 -M do 172.16.0.2
MULTIPLE DEFAULT GATEWAYS
See this document linux-two-interfaces-same-subnet.pdf
This is done by means of iproute2 and creating a new table, that should be use for the traffic coming From and To that network.
- Modify /etc/iproute2/rt_tables and point to the name of that new table
- Set the following default and local table:
ip route add 10.10.0.0/24 dev eth1 src 10.10.0.10 table rt2 ip route add default via 10.10.0.1 dev eth1 table rt2 * Set the following routing Rules (not route entries!): ip rule add from 10.10.0.10/32 table rt2 ip rule add to 10.10.0.10/32 table rt2 ip route list table all ip rule list
Needed in cluster members behind a load balancer because there's a snip, ie the load balancer is also source natting the traffic so the return flow is naturally sent back to the SNIP.
If the above doesn't fulfill, basically we require to add a table for that particular interface which (table) is checked before the global default one, sending the traffic back that interface!
To make it persistent, we use locations below https://unix.stackexchange.com/questions/354854/how-do-i-create-centos-7-persistent-policy-based-routing-rules?rq=1:
/etc/sysconfig/network-scripts/route-ethX /etc/sysconfig/network/scripts/rule-ethX /etc/iproute2/rt_tables
IPTABLES NOTES
kubernetes_networking_deep_dive.pdf
There are four types of tables. There are 5 types of 'hooks' (netfilter hooks in the linux networking stack). Chain is the same as ACL ~ table.
iptables -S # by default it shows the FILTER TABLE iptable -S -t nat # to show the other most famous table nat iptables -I INPUT 1 -s 172.18.61.145/32 -j DROP/ACCEPT # this adds a rule on top iptables -A INPUT 1 -s 172.18.61.145/32 -j DROP # this adds a rule at the end iptables -D INPUT -s 172.18.61.145/32 -j DROP # to delete a rule
TABLES:
- filter
- nat
- mangle (adjust ip headers like ttl)
- raw (bypass connection tracking)
HOOKS (in the linux packet pipeline)
- prerouting (after in-nic but before fib)
- input (between fib and local process)
- forward (fib)
- output (between local process and fib)
- postrouting (after fib but before out-nic)
Not all hooks invoke all tables , so less than 25 combinations:
- filter table is only used by input, forward and output hooks
- nat table is only used by forward,pre and post hooks
/etc/sysconfig/iptables
iptables-restore /etc/sysconfig/iptables
Classical view:
iptables -L -v
Save and apply (debian):
iptables-save iptables-restore /etc/iptables/rules.v4
If we want a view of the POSTROUTING
iptables -t nat --list * Examples: [[https://www.thegeekstuff.com/2011/06/iptables-rules-examples/?utm_source=feedburner]] * Flags/options: [[http://vault.centos.org/3.6/docs/html/rhel-rg-en-3/s1-iptables-options.html]]
Example: to access from one laptop to the .122 subnet of the KVM hosts:
In the origin, we setup a more specific route for the unetlab instances:
ip route add 192.168.122.128/25 via 192.168.0.5 dev wlp3s0
In the destination, we need to do source natting to the kvm gateway so the packets can return. We also delete the default reject for the icmp forward in the iptables.
; This only for the 1st install yum install iptables-services touch /etc/sysconfig/iptables touch /etc/sysconfig/ip6tables ; This to enable iptables for the session systemctl stop firewalld systemctl start iptables ; be sure this line is commented out in /etc/sysconfig/iptables A FORWARD -j REJECT --reject-with icmp-host-prohibited
This is the bit doing the one to one SNAT (in destination):
iptables -t nat -A POSTROUTING -o virbr0 -s 192.168.0.11 -j SNAT --to-source 192.168.122.1
NFTABLES (RHEL8 OL8):\
https://www.redhat.com/en/blog/using-nftables-red-hat-enterprise-linux-8
FIREWALLD
sudo firewall-cmd --state firewall-cmd --get-default-zone firewall-cmd --get-active-zones firewall-cmd --list-all # rules in a zone
redirec/Open ports/sorce add/deny all end:
firewall-cmd --zone=public --change-interface=eno3 firewall-cmd --zone=public --add-port=5201/tcp --permanent firewall-cmd --add-forward-port=port=8888:proto=tcp:toport=80:toaddr=192.168.100.217 --permanent firewall-cmd --add-forward-port=port=32751:proto=tcp:toport=32751:toaddr=192.168.100.217 --permanent firewall-cmd --zone=public --permanent --delete-rich-rule="rule family='ipv4' source address='148.64.57.10' reject" firewall-cmd --zone=public --permanent --add-rich-rule="rule family='ipv4' source address='82.46.188.134' accept" firewall-cmd --reload
BRCTL COMMANDS
brctl show brctl addbr br0 brctl addif br0 eth0
To disable the network bridge, just shut down the interface:
ip link set <name-of-the-bridge> down
INSTALL UNETLAB-EVENG INSTANCE IN OVH
nweserver.txt
INTERFACE MANAGEMENT (OLD IFCONFIG) To see the meaning of the flags in 'ip link show' check the following man page:
man 7 netdevice
NOTE: LOWER_UP means there is a signal at the physical level (i.e. something active is plugged in the network interface). NO-CARRIER, is the exact opposite: no signal is detected at the physical level. To remove ip address:
ip addr del 192.168.50.5/24 dev eth1
To enable/disable interface
ip link set eth1 up
SS (FORMER NETSTAT)
External Link
ss -ta # All tcp, udp and unix connections (ss with nothing lists all connections)
To view only tcp or udp or unix connections use the t, u or x option.
ss -t # Only established
To see the traffic to/from the local haproxy:
tcpdump -s0 -A -nn -N -i lo port 8906 | grep GET
See difference between host, URL and directory path:
This URL: http://www.cisco.com/pub/uploads/image.jpeg” translates into the following headers:
GET /pub/uploads/image.jpeg HTTP/1.1 Host: www.cisco.com
-H : Extra header to include in the request when sending HTTP to a server
-s : Silent or quiet mode. Don't show progress meter or error messages
-w : Make curl display information on stdout after a completed transfer. The format is a string that may contain plain text mixed with any number of variables.
-o Write output to <file> instead of stdout.
-k ignores ssl errors (case cert is not ok.
-v verbose
curl 'http://10.8.9.199:81/standard/channels-json.fcgi?url=www.mycompany1.co.uk' -H 'Host: ept.gscontxt.net'
for host in $(get_machines_in_a_cluster.sh atom |grep -v old); do curl -s -w "$host time_appconnect=%{time_appconnect} time_connect=%{time_connect} time_namelookup=%{time_namelookup} time_pretransfer=%{time_pretransfer} time_redirect=%{time_redirect} time_starttransfer=%{time_starttransfer} time_total=%{time_total}" -H 'Host: trinitymirror.mycompany1.co.uk' http://$host/northeast/channels.cgi?url=http://www.chroniclelive.co.uk/ -o /dev/null; echo ''; done
Through the haproxy (own ip) or just via the local IP:
curl -s -w 'DNS time: %{time_namelookup}; Connect time: %{time_connect}; StartXfer time: %{time_starttransfer}; Total time: %{time_total}\n' "http://127.0.0.1:8920/multizone/channels-json.fcgi?url=https%3A//parade.arkadiumarena.com/games/sudoku"```
curl -H 'Host: mediamath-prebid-nj.gscontxt.net' -s -w 'DNS time: %{time_namelookup}; Connect time: %{time_connect}; StartXfer time: %{time_starttransfer}; Total time: %{time_total}\n' "http://127.0.0.1/multizone/channels-json.fcgi?url=https%3A//parade.arkadiumarena.com/games/sudoku"
Access time from two different points:
curl -o /dev/null -s -w 'Establish Connection: %{time_connect}s\nTTFB: %{time_starttransfer}s\nTotal: %{time_total}s\n' https://sevenwest.gscontxt.net/main/channels-json.cgi?url=https%3A%2F%2F7news.com.au%2Fnews%2Fdisaster-and-emergency%2Fhomes-saved-as-nsw-bushfire-threat-eases-c-414332
Establish Connection: 0.288s
TTFB: 0.823s
Total: 0.824s
curl -s -H "host: yeah" lhttp://terra.es > /dev/null
See this External Link
curl -H "Host: moat.mycompany1.co.uk" "http://moat-backend.mycompany1.co.uk/multizone/channels-ss-json.cgi?zone=moat_standard&url=http:%2F%2Fmoat-adblock.mycompany1.co.uk/unsafe01.html"
TEST PORTS With curl:
nc -z -v -w1 google.es 443 nc -z -v -w1 google.es 808 nc -z -v -w1 google.es 80
With telnet:
cat > escape ^] end file telnet google.com 80 > test_myhost1 < escape telnet google.com 443 > test_myhost2 < escape
HTTP PROTOCOL
-
- host : The domain name of the server (for virtual hosting), and the TCP port number on which the server is listening. The port number may be omitted if the port is the standard port for the service requested.#
- …
- methods
- status codes
- Connection: keepalives : this was introduced in 1.1 and allows to reuse the tcp connection for more than one request.
https://en.wikipedia.org/wiki/HTTP/2
DIG NOTES
http://www.thegeekstuff.com/2012/02/dig-command-examples/
dig +noall +answer google.de dig +trace rule11.tech dig -x 209.132.183.81 +short # reverse records dig @9.9.9.9 redhat.com # for a specific dns server dig yahoo.com +short dig 147.in-addr.arpa. ns
BIND
host -taxfr dc.mycompany1.co.uk marrow # to pull a zonefile host -tsoa dc.mycompany1.co.uk 10.8.8.42 |grep SOA # for communication inter master slave DNS servers
—- HOST (DNS queries) To query NS record type for the domain cyberciti.biz, enter:
host -t ns cyberciti.biz
his makes host perform a zone transfer for zone name. Transfer the zone printing out the NS, PTR and address records (A/AAAA). If combined with -a all records will be printed.
host -al mycompany1.co.uk. marrow.cambridge.mycompany1.co.uk
What is my ip from cli:
dig +short myip.opendns.com @resolver1.opendns.com curl ipecho.net/plain
Show the status of modules in the Linux Kernel
lsmod
List all PCI devices
lspci
- BDF or B/D/F, as abbreviated from bus/device/function
- Bus number: pci specification allows up to 256 buses. The bus number is the address for the device in the PCI world.
- Device number: It can be 32 devicesi in each bus
- Function number
- Capabilities
https://en.wikipedia.org/wiki/PCI_configuration_space
List USB devices
lsusb
To try to fix the blurtooth issue, check in this link if its trying to load a .cab or .dfu file and see if we can find it. Don't think is the case but let's gather some info:
http://ubuntuforums.org/showthread.php?t=2274915&page=2
MINICOM (TERATERM FOR WINDOWS)
dmesg # Write down /dev/ttyUSBX node and chmod 777 minicom -s > Serial port setup # Be sure option A has: /dev/ttyUSBX Save setup as dfl Exit
—> Control-A, then E turns on local echo in minicom 2.00.0
Opengear:
After resetting the unit:
- Baud Rate: 115200 ; Data Bits: 8N1 ; Flow Control: None
- Credentials: root default
- ~. - Exit pmshell
Juniper:
- Bits per second: 9600
- Data bits: 8
- Parity: None
- Stop bits: 1
- Flow control: Xon/Xoff
CISCO TERMINAL SERVER (eg ISR4431/K9)
- Disconnect session:
- control-shift-6 then x to exit (puts the session in the background, you need to clear it afterwards!)
- “sh sess” + number get back into the box again
- clear line <line number>
- Ctrl-S and then Ctrl-Q # NOTE: if console doesn't respond, works for Arista switches at least
NOTE: For Arista devices, if console doesn't respond, issue this sequence from OOBML: Ctrl-S and then Ctrl-Q
Example mappings OCTAL NIM interface to tcp port:
TS1# OCTAL NIM2 12 0/2/12 38 TTY <--> 2038 TS2# OCTAL NIM2 9 0/2/9 35 TTY <--> 2035
OPENGEAR
To use the CLI, check this guide: External Link
To reset the Opengear's LAN setting from console, use this link
Configuring new interfaces: External Link
If the 7200 has ethernet switch ports, ie has an E in its name, like IM7216-2-24E, then ports are defined like this (UI):
- OOB NET2 - eth1
- net1-eth0 is WAN
- net3-eth2 is the mgmt-switch.
## In the opengear: https://opengear.zendesk.com/hc/en-us/articles/216371123-Configuring-from-the-command-line
This is to configure that port in the opengear. replace the X
config -g config > /tmp/config.old # SAVE BEFORE CONFIG!! config -s config.ports.portX.label fx2-X config -s config.ports.portX.loglevel 0 config -s config.ports.portX.mode portmanager config -s config.ports.portX.parity None config -s config.ports.portX.pinout X1 config -s config.ports.portX.protocol RS232 config -s config.ports.portX.speed 115200 config -s config.ports.portX.stop 1 config -s config.ports.portX.syslog.facility Default config -s config.ports.portX.syslog.priority Default config -s config.ports.portX.terminal vt220 config -s config.ports.portX.unauthssh on config -s config.ports.portX.webshell on config -r serialconfig # apply settings 1 config -a # apply settings 2 # This is to remove unwanted spaces in the xml file: cp /etc/config/config.xml /etc/config/config.bak # and then run the following commands: sed -i '$d' /etc/config/config.xml echo "</config>" >> /etc/config/config.xml config -a
MORE INFORMATION/PROCEDURE INITIAL CONFIG:
Introduce line send delay or it will fail when pasting long configs. Use paramiko: https://pynet.twb-tech.com/blog/python/paramiko-ssh-part1.html
Configure a new box (having meraki with internet access)
connect NET1 to meraki port
Check dhcp address assigned to NET1
ssh and configure lan interface (with other meraki reserved IP)
config.interfaces.lan.address x.x.x.x config.interfaces.lan.ddns.provider none config.interfaces.lan.device.type marvell config.interfaces.lan.gateway y.y.y.y config.interfaces.lan.ipv6.mode stateless config.interfaces.lan.media Auto config.interfaces.lan.mode static config.interfaces.lan.netmask 255.255.255.0 config.interfaces.oobfo.disabled on config.interfaces.wlan.disabled on config -a
move link to NET1>NET2
move link to NET1 connect to OOB
configure lan interface
config.interfaces.wan.mode static
add root user
config -s config.users.user1.description=root config -s config.users.user1.groups.group1=admin config -s cconfig.users.user1.ssh_pubkey_only=off config -s cconfig.users.user1.username=root config -s cconfig.users.user1.plaintext_password=default
Change default password
/etc/scripts/user-mod -P
Rest of the users, template:
config.users.user4.description ChrisG config.users.user4.groups.group1 admin config.users.user4.ssh_pubkey_only on config.users.user4.username chris_good
Add the user keys:
cat /etc/config/users/chris_good/.ssh/authorized_keys
Be sure matches with the actual number of users configured
config.users.total 6
Apply rest of the config
This configures all ports at once:
for (( PORT=48 ; PORT > 0 ; PORT-- )) do config -s config.ports.port$PORT.charsize=8 config -s config.ports.port$PORT.dtrmode=alwayson config -s config.ports.port$PORT.flowcontrol=None config -s config.ports.port$PORT.label=device-$PORT config -s config.ports.port$PORT.loglevel=3 config -s config.ports.port$PORT.mode=portmanager config -s config.ports.port$PORT.parity=None config -s config.ports.port$PORT.pinout=X2 config -s config.ports.port$PORT.protocol=RS232 config -s config.ports.port$PORT.speed=9600 config -s config.ports.port$PORT.ssh=on config -s config.ports.port$PORT.stop=1 config -s config.ports.port$PORT.syslog.facility=Default config -s config.ports.port$PORT.syslog.priority=Default config -s config.ports.port$PORT.terminal=vt220 done config -a
USERS CONFIGURATION:
- humans: 'admin' (tick only that role, nothing else!)
- rancid: 'users' (tick only that role, nothing else!)
EXTENDING THE OPENGEAR:
- Option 1: Serial port cascade.
If your only reason to use this is accessing serial ports then you can simply add the ports from the secondary unit to primary using cascading.The unit A will show 96 ports and can manage all of them.
https://opengear.zendesk.com/hc/en-us/articles/216373643-Setting-up-serial-port-cascading-or-clustering
- Option 2: Daisy Chain using Bridge Interface
You can enable Briding on unit A which turns Net1 and Net2 into a single shared port. This will allow you to use the device as a simple switch to let unit B access the gateway through it.Unit-A Net2 Plugged to Unit-B Net1. Unit-A Net1 Connected to the Switch/Gateway/Router https://opengear.zendesk.com/hc/en-us/articles/216372963-Aggregating-network-interfaces-with-bridging-bonding
- Option 3: Unit A handling routing
You can connect Net 2 of Unit A to Net 1 Unit B. Configure the IP for Unit-B WAN/Network interfaces with Unit-A Net-2 Address as the gateway.
Then You enable Forwarding and Masquerading on unit A from Management Lan to Network Interface.
GUI > System > Firewall > Managemenet Lan > Check Network Interface.
OPENGEAR AND SSL/TLS:
- To add a wildcard cert. todo
- To disable tlsv1.0 and tlsv1.1
config -s config.https.customciphers="ALL:-aNULL:-eNULL:-TLSv1" -r ssl
DELL CMC / iDRAC CONFIGURATION:
Initial configure dell server CMS from serial:
- Once we have access to the cmc, apply this for the cmc network settings. default user root ; password calvin.
- 115200 baud. 8 data bits, 1 stop bit, and no parity.
- To recover ILOM root password:
racadm getniccfg
racadm getsysinfo racadm config -g cfgLanNetworking -o cfgNicIpAddress <static ip> racadm config -g cfgLanNetworking -o cfgNicGateway 10.8.11.254 racadm config -g cfgLanNetworking -o cfgNicNetmask 255.255.254.0 racadm config -g cfgLanNetworking -o cfgNicUseDHCP 0 racadm config -g cfgLanNetworking -o cfgDNSServersFromDHCP 0 racadm config -g cfgLanNetworking -o cfgDNSServer1 8.8.8.8 racadm config -g cfgLanNetworking -o cfgDNSServer2 8.8.4.4 racadm racreset
To modify parameters from inside the own blade (linux) issue:
racadm setniccfg -s 10.8.11.210 255.255.254.0 10.8.11.254
For stuck sessions, do
racadm getssninfo racadm closessn -i <id>
FX2 PORTS
See : https://ecauk.com/files/2016/07/Dell-PowerEdge-FX2-Chassis-Owners-Manual.pdf
- Page 27 I/O module port mapping
OTHER COMMANDS:
http://en.community.dell.com/techcenter/systems-management/w/wiki/3205.racadm-command-line-interface-for-drac
http://www.gooksu.com/2015/04/27/racadm-quick-dirty-cheatsheet/
To see configuration (then inside will be the objects):
racadm getconfig -g ... cfgLanNetworking cfgIPv6LanNetworking cfgCurrentLanNetworking cfgCurrentIPv6LanNetworking cfgRemoteHosts cfgUserAdmin cfgEmailAlert cfgSessionManagement cfgSerial cfgNetTuning cfgOobSnmp cfgTraps cfgRacTuning cfgRacSecurity cfgActiveDirectory cfgStandardSchema cfgLDAP cfgLDAPRoleGroup cfgChassisPower cfgServerInfo cfgKVMInfo cfgAlerting cfgLcdInfo cfgLocation cfgThermal
To configure the object:
racadm config -g cfgRemoteHosts -o cfgRhostsSyslogPowerLoggingInterval 1
To open a case, email dell with logs from cmc:
racdump dumplogs
- To toggle video among the quarter-height blades installed in a sleeve:
- Press <Print Screen> to launch the OSCAR interface (iKVM GUI).
- Select the slot where the sleeve and blade are installed, and press <Enter> to exit the OSCAR interface.
- Press <Scroll Lock> twice in quick repetition to toggle between the quarter-height blades installed in the sleeve.
DELL CHASSIS BLADES.
http://bladesmadesimple.com/2010/11/dell-m1000e/
Pg 53 here: https://i.dell.com/sites/doccontent/shared-content/data-sheets/en/Documents/PowerEdge_M_Series_Blades_IO_Guide.pdf
Port mapping: On two port cards, port 0 would go to I/O Module Bay 1 and port 1 would go to I/O Module Bay 2. On four port cards, the even # ports (0 and 2) would go to I/O Module Bay 1 and the odd # ports (1 and 3) go to I/O Module Bay 2.
Important to know IOM and mezzaninee (blade) nic configuration.
Current project:
https://downloads.dell.com/manuals/all-products/esuprt_ser_stor_net/esuprt_networking/esuprt_net_blade_intercnts/force10-mxl-blade_user%27s%20guide2_en-us.pdf
MXL 10/40GbE
thirty-two 1/10-Gigabit Ethernet internal ports.
To change uplink from
4x10GbE to 40GbE
no stack-unit port portmode quad
Ports are numbered 33 and 37
mycompany2(R) Integrated Lights Out Manager - ILOM COMMANDS Version 4.0.3.21 r125986
credentials: root changeme
Serial settings: 8N1: eight data bits, no parity, one stop bit ; 9600 baud ; Disable hardware flow control (CTS/RTS)
mycompany2 X6-2 is 115200 baud
more info about console credentials reset : External Link
show /SP/network # Show ilom settings cd /SP/network set pendingipdiscovery=static set pendingipaddress=10.8.183.106 set pendingipnetmask=255.255.255.0 set pendingipgateway=10.8.183.254 set commitpending=true set state=enabled show /SP set /SP hostname=Lab2-System1 show /SP/network # Show ilom settings set /SP/users/root password show /SP/network reset /SP # to reset the CMC
Enable 10G ports (mycompany2 X7-2 servers:
set /HOST/network/ set active_media=SFP28 # + power cycle the host (from UI) OR auto_media_detection=disabled reset /System linux# reload # (if still doesn't work)
DELL CHASSIS SWITCHES PE-FN-410S-IOM
cli guide External Link
show link-bundle-distribution port-channel 128 # to see status of the uplink Po
TIME MANAGEMENT: NTP AND CHRONY
ntpq -p # to see the ntp status * refid column shows the current source of synchronization * st column reveals the stratum, t the type (u = unicast, m = multicast, l = local, - = don't know) * poll the poll interval in seconds * when column shows the time since the peer was last heard in seconds, while the * reach column shows the status of the reachability register (see RFC-1305) in octal * Offset is the clock time difference between the peers or between the master and client. This value is the correction that is applied to a client clock in order to synchronize it. * Dispersion, reported in seconds, is the maximum clock time difference that was ever observed between the local clock and server clock. Expect it high when clocks are syncing.
chronyc sources chronyc tracking
DELL R740 RISERS AND PCI CONFIGURATION:
dell-r740_risers_configuration.docx
VRRP in linux, with keepalived\\:
/etc/keepalived/keepalived.conf
global_defs {
notification_email {
sysadmin@mycompany1.co.uk
}
notification_email_from sysadmin@mycompany1.co.uk
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id RSYSLOG2
}
vrrp_instance RSYSLOG-1-2 {
interface bond0 # interface to monitor
state BACKUP
virtual_router_id 40 # Assign one ID for this route
priority 100 # 101 on master, 100 on backup
authentication {
auth_type PASS
auth_pass 5291
}
virtual_ipaddress {
10.8.9.104 # syslog-vip.dc.mycompany1.co.uk # DO NOT REMOVE THIS COMMENT
}
track_script {
checkRsyslog
}
}
vrrp_script checkRsyslog {
script /etc/keepalived/keepalived_check_rsyslog
interval 2
weight 2
}
UNETLAB When installing unetlab in KVM, be sure that:
- The host has nested virtualization enabled: External Link. Just be sure grub is modified correctly so the change survives reboot (note that the commands to apply the changes in /etc/default/grub change from BIOS to UEFI systems. To allow nested virtualizarion, needless to say, your host need to allow virtualization. Check BIOS/UEFOI for this.
- KVM is copying the CPU config from host to guest. KVM virtual manager, CPU section , tick 'copy host cpu configuration.
- To add ansible and cloud interface, see entry in softbackbone
- To integrate telnet, vnc and Wireshark see: External Link
- Set up Telnet, VNC, and Wireshark. See section in this link http://www.brianlinkletter.com/how-to-set-up-the-eve-ng-network-emulator-on-a-linux-system/ .
UNETLAB INTEGRATION (EVE-NG INTERGRATION)script: https://github.com/SmartFinn/eve-ng-integration & also this zip with script: unetlabtricks.zip
- Follow the instructions above ^^ (ie compile it).
- Edit the python script and be sure it invokes wireshark correctly (in Fedora change wireshark > wireshark-qt)
- Configure ssh keys with the remote eve-ng instance
- If FF shows open link greyed out, just open Preferences> applications > capture , and point to to 'use eve-ng-integration' (do not ask).
UNLOCK EVE IMAG (FROM 'VENDOR'):
- import ovf
- press esc while loading
- append 'single' to line starting with linux
- Follow 2nd set of steps here: http://www.noah.org/wiki/Single_User_Mode
- Then ssh to the vm and reset admin password:
echo "DELETE from users where username = 'admin';" | mysql --host=localhost --user=root --password=eve-ng eve_ng_db
echo "INSERT INTO users VALUES ('admin',NULL,'root@localhost',-1,'Eve-NG Administrator','85262adf74518bbb70c7cb94cd6159d91669e5a81edf1efebd543eadbda9fa2b',NULL,'','admin','',1);" | mysql --host=localhost --user=root --password=eve-ng eve_ng_db_
PORT FORWARDING (REMOTE LAB):
firewall-cmd --add-masquerade firewall-cmd --add-forward-port=port=8888:proto=tcp:toport=8888:toaddr=192.168.122.180
EVE-NG UNETLAB TROUBLESHOOTING:\\:
/var/log/syslogs
QEMU BASED INSTANCES:
- Download ove from http://wedding.arakchi.net/Networks/vMX/
- root root123
- qemu-img convert -f vdi -O qcow2 vMX-14.1R1.10.vdi hda.qcow2
- /opt/unetlab/wrappers/unl_wrapper -a fixpermissions
- Then be sure the instance is running last qemu version (>2.9) , 4098G memory and 2 CPU
CONNECT INSTANCE TO THE OUTSIDE WORLD:
- There's no need to create another interface in the unetlab install (virtual manager level). In fact this would prevent unl to start with 'waiting for network'
- Right click in the diagram > network > pnet0 (note this is not creating pnet0, pnet0 was already there as the unl vm interface, the one we were currently using to access the instances via telnet or guu).
- Connect cloud to instance in diagram.
- The IOL instances will have a vunlXXX created, we need to add them to the bridge where pnet. Example below:
brctl delif pnet1 vunl0_37_16 brctl addif pnet0 vunl0_37_16 brctl show
CAPTURE PACKETS IN UNETLAB (LAB INTERFACES):
- Check interface id:
- Shut down the instance, right click and top of window (send line) ID1. Check the id of the interface on the other end (ID2)
brctl show | egrep -B 1 -A 1 vunl0_ID_
- Check in what bridge we have vunl0_ID1 and vunl0_ID2. That's the bridge where we need to capture the traffic. Use the interface in any of each ends:
tcpdump -nni vunl0_52_0
RANCID NOTES
Tshoot:
- Note that rancid can use different cipher suite and key-exchange methods than the linux host is running it.
- Check rancid logs
- Test jlogin (or nslogin or ograncid) with:
rancid$ /usr/libexec/rancid/**jlogin** csw1.la1.mgt.mycompany1.co.uk # note is under rancid user. do not add user to the ssh! rancid$ ssh -i /var/rancid/.ssh/rancid_rsa -c 3des -x -l rancid csw1.la1.mgt.mycompany1.co.uk # this is to force a specific cipher ssh -v -i /var/rancid/.ssh/rancid_rsa -x -l rancid csw1.la1.mgt.mycompany1.co.uk # Removing the forced 3des cipher works. Has the juniper upgrade changed the accepted ciphers?
Some config here:
/usr/libexec/rancid
And another bit here:
/var/rancid
This is to enable non standard ports for certain devices:
[rancid@nms]$ cat .cloginrc
add password fw0* {} {}
add user * rancid
add password * password
add identity * /var/rancid/.ssh/rancid_rsa
add method csw3.la1.mycompany1.co.uk {ssh:1023}
add method csw4.la1.mycompany1.co.uk {ssh:1024}
add method {ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no}
add method {ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no}
add method * ssh
add noenable fw0* {0}
add cyphertype fw0*.dc.mycompany1.co.uk aes256-cbc
OBSERVIUM SHOULD ADD ENTRIES IN:
/var/rancid/observium/router.db rancid$ /opt/observium/scripts/generate-rancid.php > /var/rancid/observium/router.db
# CREDENTIAL DETAILS IN:
/var/rancid/.cloginrc
# FORCE RANCID RUN:
rancid$ /usr/bin/rancid-run
# OBSERVIUM RANCID TROUBLESHOOT (SVN NOT WORKING, REFLECTED IN THE UI)
https://jira.mycompany2datacloud.com/browse/SYS-3702
~In observium.dc.mycompany1.co.uk cd /var/rancid/observium/configs/ svn cleanup touch /var/rancid/observium/.svn/pristine/2e/2e038b490aa4a3d1fa48d17473523f55063fafa7.svn-base svn cleanup svn update --set-depth empty svn update --set-depth infinity svn status -v svn add --force . svn ci -m "" ~ Then cd /var/rancid mv observium observium.bak3 /usr/libexec/rancid/rancid-cvs php /opt/observium/scripts/generate-rancid.php > /var/rancid/observium/router.db svn update svn cleanup
To make dangerous changes (and avoid being kicked out):
(ip route add 10.0.0.0/8 via 10.20.3.5 dev eth0 ; echo 'successful'; sleep 30 && ip route del 10.0.0.0/8 via 10.20.3.5 dev eth0)
CUSTOMIZE COMMANDS TO PULL DATA FROM DEVICES:
we need to modify the file in rancid.types.base. DO IT UNDER RANCID USER! https://www.cryptomonkeys.com/2013/09/rancid-junos/:
/etc/rancid
JIRA
IPERF NOTES (IPERF3)
https://openmaniak.com/iperf.php
By default, the Iperf client connects to the Iperf server on the TCP port 5201 and the bandwidth displayed by Iperf is the bandwidth from the client to the server.
Iperf measures both TCP and UDP bandwidth performance.
On the Linode you wish to test, launch iPerf in server mode:
iperf3 -s # to activate the listening SERVER iperf3 -P 20 -c $server_ip # 20 parallel flows iperf3 -P 20 -u -c $server_ip # to use udp , as tcp sometimes gives wrong results iperf -P 20 -c 10.8.1.120 -u -b 500m -t 15 -i 1 -l 1500 # -b is target address per each of the x20 flows ;-t 15 seconds transmitting ; to get periodic bandwidth, jitter, and loss reports iperf3 -c
If what we need to masure is latency and jitter we can use netperf https://github.com/HewlettPackard/netperf:
sudo apt-get install -y netperf
*LLDPD LINUX ( LLDP ) If we want to disable specific TLV, note that the TLV needs to go with commas:
[lldpcli] # unconfigure lldp custom-tlv oui 00,27,33 subtype 2 # for OUI DCBX (0x001b21) ; Subtype (2)
TRANSMISSION-CLI* su - transmission
# to change the config stop th e daemon BEFORE: service transmission-daemon stop vim /var/lib/transmission-daemon/info/settings.json systemctl start transmission-daemon.service .config/transmission-daemon/settings.json sudo su --shell /bin/bash --login debian-transmission transmission-remote -a "magnet or torrent here" transmission-remote -l # to review status transmission-remote -t <torrent ID> --remove-and-delete transmission-remote --torrent all --start
Transmission re-add corrupted:
transmission-remote -l info=$(transmission-remote $TRANSMISSION_ARGS -t 19 -i) magnet=$(echo "$info" | grep magnet | sed 's/Magnet: //') transmission-remote -t 19 --remove-and-delete transmission-remote -a $magnet
Blocklist: https://github.com/sayomelu/transmission-blocklist
TC NOTES TRAFFIC CONTROL:
- http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm (in 7. Understanding statistics)
- This is a nice (and modern) tc frontend for linux (unrelated to the ISP): https://github.com/tylertreat/Comcast
FQ NOTES
Implement BBR TCP congestion control:
/etc/sysctl.conf net.core.default_qdisc=fq # queue discipline fair queue net.ipv4.tcp_congestion_control=bbr # congestion algorithm
NMCLI COMMANDS:
nmcli device show nmcli connection show nmcli device wifi list nmcli connection show nmcli connection down pandas3 nmcli connection up id panda2 nmcli device wifi connect pandas3 password xxxx ifname wlp2s0 nmcli connection down id panda3
—-
MRTG
/etc/mrtg/snmpvar.{cf|def}
/etc/mrtg/ plus the actual mrtg instrumentation
To change the properties of any particular 'definition', graph, device polled:
You just edit the definition in the mrtg config directly. For instance you'd change
Target[10.8.11.126_FPCTemps]: .1.3.6.1.4.1.2636.3.1.13.1.7.7.5.0.0&.1.3.6.1.4.1.2636.3.1.13.1.7.7.6.0.0:R8HhBos- dn3H6B@10.8.11.126:::::2
At the top of mrtg files you'll generally get a definition of what created it
# /usr/bin/cfgmaker --snmp-options=:::::2 public@10.8.11.126 --zero-speed=125000000 --no-down
You can run that command and you can see if it is getting stuff as you twiddle the options
Example of creating a snmpv3 definition:
/usr/bin/cfgmaker --global 'LogFormat: rrdtool' --global 'WorkDir: /var/www/mrtg' --global 'Options[_]: bits, growright' --global 'EnableIPv6: no' --output /etc/mrtg/sw-e12.cfg --enablesnmpv3 --snmp-options=:::::3 -contextengineid=0x80001f8880b0c601061e21d95700000000 --username=ODC --authprotocol=sha --privprotocol=aes --authpassword=XXXXXXXXXXX --privpassword=XXXXXXXXXX sw-e12 rm -f /var/www/mrtg/sw-e12*log rm -f /var/log/mrtg/sw-e12*log env LANG=C /usr/bin/mrtg /etc/mrtg/sw-e12.cfg
APACHE BENCHMARK TOOL:
ab -n 100 -c 1 -k #which will run 100 queries sequentially using keepalives
PING NOTES:
ping -M do -s 8972 daffy01.lhr.oci.mycompany1.co.uk | while read pong; do echo "$(date): $pong"; done # To have timestamps
Jumbo frames in Linux:
sysctl -a | grep -i mtu # verify capable ping -M do -s 8972 [destinationIP] # test
MRT NOTES:
mtr -rw -bc 3 203.190.181.15 > mtr-report # nice report with three pings mtr -b --tcp 203.190.181.155 # tcp syn mtr -o "LSDR NBAW JMXI" 216.58.223.78 # to add jitter fields
WHOIS NOTES:
whois -h whois.apnic.net 203.190.181.0 whois -h whois.ripe.net 148.64.56.0 whois -h whois.arin.net 185.89.204.0
Remember to have to IRR in shape to prevent unexpected blockholing or inter provider routing problems!:
https://fcix.net/whitepaper/2018/07/14/intro-to-irr-rpsl.html
X11VNC VNCSERVER NOTES
- install it (step 1 to 4): https://tecadmin.net/setup-x11vnc-server-on-ubuntu-linuxmint/
- rc-local for autostart. Place this in /etc/rc.local and change permission chmod +x /etc/rc.local
#!/bin/bash su - camarreal -c "x11vnc -display WAIT:0 -forever -bg" > /dev/null 2>&1 &
- enable autostart rc.local: https://linuxmedium.com/how-to-enable-etc-rc-local-with-systemd-on-ubuntu-20-04/
MULTICAST IN LINUX:
LINUX INSTANCES:
linux-tinycore-6.4.tar.gz - 512mb - 1cpu - for root type : sudo su
To connect tinycore to the host, eve-ng cloud 0 and this is eveng ' ip a a 10.8.88.1/24 dev pnet0'
