MDT (Model Driven Telemetry): Replaces SNMP
What is gRPC (streaming telemetry)
- Google Remote Procedure call (remote call on the device) . Is an open source alternative/improvement to classic restAPI. It adds consistency, also uses binary messages instead of json text so it's more efficient traffic-wise. Uses standard language: protobuf which is supported by python among others.
- protobuf 'file' define what RPCs are available. Is provided by the vendor and abstracts the way to interact to the system.
- We convert or compile 'protobuf' to python libraries. 'Protoc' creates a library that you can start using straight away in your code
- The messages are binary but we can do marshaling and see the key-value pairs with 'kgbb'
- It uses http2 by default (port 50051). With a single channel it can send a bunch of requests. So only one http handshake is needed.
- All is defined in the client (jti-client in our case) (define how many channels, etc)
gNMI
See this insight on prometheus-grafana-exporters and gnmi: monitoring_cisco_networks_with_prometheus_grafana_and_exporters_fintech_focus_.pdf
- Provides 1) Network Management and 2) Streaming telemetry
- Uses HTTP/TCP so reliable compared to snmp
- And also extremely light. It uses RPCs more efficiently than netconf (like it were issued in the target machine itself) See comparison
- gRPC Network Management Interface. Is a way to interact to grpc with Standard, reduced, RPCs.
- gnmi defines Four RPCs:
capabilities(“show me what you can offer”)get(like a netconf get)set(like a netconf set, also to update and delete)subscribe( hook to a stream )
- Junos in particular uses the JTI client which includes gNMI : Junos Telemetry Interface Client (jtimon) is an application that can be used to collect telemetry streams from a Juniper device streaming in native format over UDP, or in OpenConfig format over gRPC.
Topology ( gNMI + Telegraf plugin)
MESSAGE QUEUING TECHNOLOGIES
- The client subscribes to the server/broker but the client can receive or send data
- Topics are labels that you publish messages to and subscribe to. They are arranged as a hierarchy
Kafka
Kafka focuses on the storage and reading of data, aiming at streaming data processing scenarios with high real-time performance
MQTT - MQ Telemetry Transport
is a lightweight, publish-subscribe, machine to machine network protocol for Message queue/Message queuing service. It is designed for connections with remote locations that have devices with resource constraints.
sudo apt install mosquitto mosquitto-clients # creates the broker. This can be a simple raspberry pi mosquitto_sub -h localhost -t test # Test subscription to a test topic (still non-existing) mosquitto_pub -h localhost -t test -m "hello world" # publishes a test message in the 'test' topic ! mosquitto_sub -h <host> -t /merakimv/<serial of device>/<topic> # subscribes
Then PMACCT (pmaccount) to correlate information and extract trends: pmacct is a small set of multi-purpose passive network monitoring tools [NetFlow IPFIX sFlow libpcap BGP BMP RPKI IGP Streaming Telemetry].
References: