Share with Your Network
Automation is the one of the keys to a successful DevOps department. There are many workflow engines, like the one in Cisco’s SecureX. This blog discusses one way to automate launching connector runs.
Why automate connector runs? One reason is because you will have vulnerability risk information at the same time each day. This provides consistent data for statistical purposes. Unburdening people from launching connector runs manually gives time to your DevOps personnel for other projects.
Introduction
There are two code examples to draw from:
- connectors_auto_start.py – scans the list of connectors and determines when to launch a connector run and is discussed in this blog.
- show_connector_status.py – scans the list of connectors and informs you when a connector would be launched.
After experimenting with the connector APIs, I created connectors_auto_start.py. The code example, show.connector_status.py, is what the experimentation left behind. It is not polished, but has some useful functions.
I decided to use the PrettyTable library for output in connectors_auto_start.py. This library is easy to use and is great for making readable output. Please use the requirements.txt to install the libraries. I’ll discuss more about PrettyTable later on.
The flow of connectors_auto_start.py is:
- Obtain a list of connectors
- For each connector, get the latest connector run
- If the criteria is met, launch the connector run
Obtaining connectors
To obtain a list of connectors, the List Connector API is invoked. The response information contains the connector name, the connector ID, the connector host address, and if the connector is currently running. Later on the host address determines whether this connector is a file based connector or not. The response is returned to the caller.
11 # Returns connector information from the List Connectors API. 12 def get_connectors(base_url, headers): 13. connectors = [] 14. list_connectors_url = base_url + "connectors" 15 list_connectors_url = f"{base_url}connectors" 16 17. response = requests.get(list_connectors_url, headers=headers) 18. if response.status_code != 200: 19. print(f"List Connector Error: {response.status_code} with {list_connectors_url}") 20. sys.exit(1) 21 22. resp_json = response.json() 23. connectors = resp_json['connectors'] 24 25. return connectors
Obtaining connector runs
With the list of connectors, we eliminate the file-based connectors. Why? Because I didn’t want to code uploading a file at this time. (Another blog perhaps). Non-base file connectors have a host associated with them.
109 # Check if connector is file based. 110 if connector['host'] is None: 111 conn_tbl.add_row([name, "is a file base connector"]) 112 continue
Here the main code obtains a list of connector runs. If a connector doesn’t have any runs, then the automation won’t try to launch it. Why? Because if it hasn’t been run at least once, it hasn’t been tested.
114 # Obtain the connector runs for a connector. 115 connector_runs = get_connector_runs(base_url, headers, id) 116 if len(connector_runs) == 0: 117 conn_tbl.add_row([name, "has no runs"]) 118 continue
The get connector runs function invokes the List Connector Runs API to obtain a list of connector runs for each connector. The connector ID is used to specify the connector.
26 # Gets the connector runs for the specified connector ID. 27 def get_connector_runs(base_url, headers, connector_id): 28 get_connector_runs_url = f"{base_url}connectors/{connector_id}/connector_runs" 29 30 response = requests.get(get_connector_runs_url, headers=headers) 31 if response.status_code != 200: 32 print(f"List Connector Runs Error: {response.status_code} with {get_connector_runs_url}") 33 sys.exit(1) 34 35 resp_json = response.json() 36 37 return resp_json
Note that the JSON return is the response since the array does not have a field value.
Launching the connector run
Two more criteria to go.The first criterion checks if the connector is already running by checking the start and end time stamps on the latest run. If the end time stamp is not available, then the connector is running. I think this is a nice feature which I discovered while experimenting, which means that this feature is not in the current API documentation.
120 # Only check the latest run 121 latest_run = connector_runs[0] 122 start_datetime = parse_time_str(latest_run['start_time']) 123 if latest_run['end_time'] is None: 124 conn_tbl.add_row([name, "still running"]) 125 continue 126 end_datetime = parse_time_str(latest_run['end_time'])
If the connector is not running, then grab the end time stamp and proceed to the second criterion which checks if enough time has elapsed since the connector ended. The elapsed time can be passed as a command line input parameter. If not specified, it defaults to 24 hours.
128 # Check if the end time was interval hours ago. 129 if (end_datetime + timedelta(hours=interval_hours)) > datetime.now(): 130 conn_tbl.add_row([name, f"has to wait {interval_hours} hours past {end_datetime}."]) 131 continue
All criteria have been met, so launch the connector with the Run Connector API. The connector run ID is returned.
39 # Starts a connector run based on the connector ID and returns a connector run ID. 40 def run_connector(base_url, headers, connector_id): 41 run_connector_url = f"{base_url}connectors/{connector_id}/run" 42 43 response = requests.get(run_connector_url, headers=headers) 44 if response.status_code != 200: 45 print(f"Run Connector Error: {response.status_code} with {run_connector_url}") 46 sys.exit(1) 47 48 resp_json = response.json() 49 if not resp_json['success']: 50 print(f"Running {connector_id} failed. Check log files.") 51 sys.exit(1) 52 53 return resp_json['connector_run_id']
The connector run ID is used in the output.
133 # Launch the connector if all tests passed. 134 connector_run_id = run_connector(base_url, headers, id) 135 conn_tbl.add_row([name, f"launched connector run {connector_run_id}."])
PrettyTable
As stated earlier, the PrettyTable library is straight-forward to use. Here is where the table is initialized.
96 # Set up the output table 97 conn_tbl = PrettyTable() 98 conn_tbl.field_names = ["Connector Name", "Status"] 99 conn_tbl.align["Connector Name"] = "r" 100 conn_tbl.align["Status"] = "l" 101 conn_tbl.border = False 102 conn_tbl.add_row(["~~~~~~~~~~~~~~", "~~~~~~"])
As you can see, there are two columns, “Connector Name” and “Status”. The “Connector Name” column is right aligned and the “Status” column is left aligned. Borders are turned off, but a row of tildes are added to separate the heading from the rows.
Rows are added with the add_row() method. The table is printed with a print() call.
138 print(conn_tbl)
Conclusion
So now you know how to use the List Connectors, List Connector Runs, and Run Connector APIs. This blog gives some ideas on how to automate the timing of launching connector runs that should be adaptable to workflow engines.
If you’re interested in playing with these samples, they’re located in theKenna Security blog_samples repo in the connectors directory. One thought would be to add code to check if a connector is running too long.
dev@kennasecurity.com
