Introduction
net-api is a documented REST API platform which returns structured data from network devices. This application is a mix of technologies and techniques and has been developed to highlight what is possible with multiple open source projects.
Table of Contents
net-api Overview
The overview of net-api is shown in the diagram below:
As shown in the diagram, net-api handles device authentication, data transformation and transport from network devices and presents the consumer of the application with a fully documented API. This approach has multiple benefits to name a few:
- Provide non-network operators access to configuration and state about the network, without the ability to make direct changes to devices.
- Provide a reliable, structured API to query, which can be leveraged from third-party systems or custom applications.
- Provide a framework which is extensible and portable to deploy in most environments. Additional operations or API calls can be developed with minimal effort.
What's under the hood?
This application is made up of the following:
- Flask web framework to serve the web application
- Connexion to handle HTTP requests and serve the SwaggerUI
- A fully documented API using Swagger UI
- Nornir automation framework for device management
- NAPALM to present structured data back from multiple vendors for certain functions
- TextFSM NTC Templates support to parse CLI output into structured data
- Genie support to parse CLI output into structured data.
- Netmiko support to send commands to Netmiko supported devices.
- Scrapli support to send commands to Scrapli supported devices.
Supported Environments
This application is only supported on:
- Python 3.6 or greater
- Linux/unix machines only
Videos
Below are some videos which have been recorded for net-api
net-api Introduction Video
In the video below, a brief introduction is given to net-api and a demonstration on it's capability with Scrapli, Netmiko, Genie and TextFSM.
Installation Videos
Below are videos for installing net-api on Python3 and Docker:
Installation/Operating Instructions
There are two methods for installing or operating net-api; using a Python virtual environment or a Docker container. The instructions for each method are described below.
Python 3.X
The most popular way of running this application is using it in a standard Python environment. To do so, please follow the options below:
- Clone the repository to the machine on which you will run the application from:
git clone https://github.com/writememe/net-api.git
cd net-api
-
Populate your Nornir inventory files:
Refer to the Nornir Inventory Documentation if you have not used Nornir before or follow the examples provided in this repository.
- Create the virtual environment to run the application in:
virtualenv --python=`which python3` venv
source venv/bin/activate- Install the requirements for
net-api
pip install -r requirements.txt
- Set two environmental variables, which are used by the application as the default credentials to login to devices:
export NORNIR_DEFAULT_USERNAME=<someusername>
export NORNIR_DEFAULT_PASSWORD=<somepassword>- Validate these environmental variables by entering the following command:
env | grep NORNIR
You should see the two environment variables set.
- Setup an environmental variable to point to the NTC templates directory for TextFSM functionality.
In the example below, the virtual environment is using python3.6. You will need to adjust this if using anything else:
export NET_TEXTFSM=$VIRTUAL_ENV/lib/python3.6/site-packages/ntc_templates/templates- Validate these environmental variables by entering the following command:
env | grep NET_TEXTFSM
You should see the environment variable set.
- Change to the
app/directory, then start the flask application:
cd app
python webapp.py- The application will now be running on TCP/5000. For example, if the server IP on which the app is running is 10.0.0.1, the application will be available on http://10.0.0.1:5000
Docker
There is an option to build this application in a Docker image and run it as a container. To do so, please follow the options below:
- Clone the repository to the machine on which you will run the Docker container from:
git clone https://github.com/writememe/net-api.git
cd net-api
- Populate your Nornir inventory files:
Refer to the Nornir Inventory Documentation if you have not used Nornir before or follow the examples provided in this repository.
- Build the Docker image, whereby
net-apiis the name of the image andlatestis an arbitary docker tag:
make build- Create a file of environmental variables, to be passed into the Docker image as it's starting up.
In the below example, the file.env-varscontains three environmental variables:NORNIR_DEFAULT_USERNAME- Used by the application as the default username to login to devices in the Nornir inventory.NORNIR_DEFAULT_PASSWORD- Used by the application as the default password to login to devices in the Nornir inventory.NET_TEXTFSM- Used to point to the NTC templates directory for TextFSM functionality
.env-vars
NORNIR_DEFAULT_USERNAME=<someadmin>
NORNIR_DEFAULT_PASSWORD=<somepassword>
# Change the `python3.6` to the version you are using in your environment.
# Generally, you shouldn't need to adjust this for the accompanying Dockerfile
NET_TEXTFSM=/usr/local/lib/python3.6/dist-packages/ntc_templates/templates- Run the docker build, passing in the environment variables in Step 3. This will expose the build on port 5000:
make run- Verify that the container is up and operational, by running
docker ps:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
fe8c0b858f18 net-api:latest "python3 webapp.py" 21 minutes ago Up 21 minutes 0.0.0.0:5000->5000/tcp quizzical_rosalind
$- The application will now be running on TCP/5000. For example, if the client IP is 10.0.0.1, the application will be available on http://10.0.0.1:5000
Example Use Cases
To give you an idea of some of the example use cases for net-api, there is an accompanying repository called net-api-tools containing code examples of some use-cases for consuming net-api.
These are intended to spark your imagination and show some really simple solutions, all based on net-api.
Design Decisions
When developing this application, there a few primary drivers for the design:
1) Portable
All projects that I develop for public consumption must be portable. This means that where possible, my project will be as portable as possible to other environments. This means any usage of "business logic" or "tribal knowledge" is not baked in.
This is also the intention of net-api-tools, where "business" examples are provided to give you an example of how to use this platform within the context of your business.
2) Extensible
This project is meant to be as extensible as you require. The project has a standard structure which can be followed to develop more API calls, or taken into a different direction.
Most of you will be concerned at the lack of security on the front end at the usage of environmental variables to set credentials. This is intentional as most people have their own credential system and authentication mechanisms within their environment. This project would be limited in extensibility if I catered to certain systems to handle those problems.
3) Useful
If I can't see this being useful for others, then there isn't really a point of developing it. I'm hoping that you fire it up in your lab environment or in some small corner of your network and give it a go. I've designed this with it being useful to others as soon as possible. The idea is that within 20 minutes, you are consuming the API and gaining value rather than tweaking environment-specific settings.
Frontend Authentication
As mentioned above, the application has no frontend authentication mechanisms. There are many ways that you can go about solving this problem, depending on your environment:
- Frontend the service with an application load balancer which handles your preferred authentication method.
- Extend Flask to perform authentication.
- Terminate behind a firewall and restrict access using firewall security methods.
Credential Management
Credentials are required to connect to the devices in the Nornir inventory. Having them defined as environmental variables allows you to inject these in from most credential management systems.
If that's not feasible or palatable, you can also review the code block below in app/net.py and adjust what the nr.inventory.defaults.username and nr.inventory.defaults.password values are set to. This is the only block of code which needs to be adjusted:
# General functions, consumed by other functions
def get_nr():
"""
Initialises a Nornir inventory from the various configuration
files
:return nr: An initialised Nornir inventory for use in other functions.
"""
nr = InitNornir(
inventory={
"options": {
"host_file": "inventory/hosts.yaml",
"group_file": "inventory/groups.yaml",
"defaults_file": "inventory/defaults.yaml",
}
}
)
# Set default username and password from environmental variables.
nr.inventory.defaults.username = env_uname
nr.inventory.defaults.password = env_pword
return nrTODO/Roadmap
This section will describe some of the items which I would like to complete and some other items which are more aspirational and subject to interest from others and time I can commit to this project:
TODO
- Add pytest tests to test backend functions.
- Add pytest tests to test API functions in online mode.
- Add pytest tests to test API functions in offline mode.
- Mark pytest tests so tests can be layered for certain use cases.
- Provide example custom pytest tests so others can utilise them.
- Add the NAPALM getters
lldp_neighbors,lldp_neighbors_detail,bgp_neighbors,bgp_neighbors_detailandenvironment.
Roadmap
- Add all other viable NAPALM getters to
net-apias per the NAPALM getters support matrix - Ensure Github Actions automates and execute backend function tests with pytest.