开源的威胁情报收集器：OSTrICa

OSTrICa是一款开源威胁情报收集器，也是一个插件模式的框架，能够可视化展示收集到的威胁情报数据。

在攻防对抗的过程中，SOC专家、事件响应人员、攻击调查人员以及网络安全分析人员需要将IoCs(Indicator of Compromise，入侵指示器)、网络流量等其它收集到的信息关联起来分析——这其实也是威胁情报的重要作用。但是，并非所有的企业都有足够的预算研发威胁情报平台。

OSTrICa是一个免费的开源的框架，使用基于插架的架构，可以让每一个人自动化地从公开的、内部的、商业的源中收集信息，并可视化显示各种类型的威胁情报数据。由专家来分析收集的情报，显示成图形格式，还可基于远程连接、文件名、mutex等，显示多个恶意软件的关联情况。

依赖库

OSTrICa自身并不依赖外部的库，安装的插件需要依赖库，比如：

BeautifulSoup, 几乎所有的插架都会用到

dnspython-1.12.0, mainly used by 主要由CymruWhois插件使用

ipwhois-0.11.1, PyWhois插架使用

python-deepviz-master, DeepViz插件使用(它需要API key) – 此时，DeepViz插件还没有向公众开放

python-whois-0.5.2, PyWhois插件会用到

pythonwhois-2.4.3, PyWhois插件会用到

requests, ThreatCrowd插件使用(请求限制根据https://github.com/threatcrowd/ApiV2)

注意：OSTrICa适用于版本>= 2.7.9的Python环境。

如何使用OSTrICa

执行main.py文件使用OSTrICa，执行help命令列出可用的命令：

> python main.py

OSTrICa v.0.5 - Open Source Threat Intellicence Collector

Developed by: Roberto Sponchioni - @Ptr32Void

write "help" for help

> help

Following options are available

domain - used to collect domains information

Example: domain=google.com or domain=google.com,yahoo.com

ip - used to collect IP information

Example: ip=8.8.8.8 or ip=8.8.8.8,173.194.68.99

md5 - used to collect MD5 information

sha256 - used to collect SHA256 information

asn - used to collect ASN information

email - used to collect email information

graph - generate a graph based on all the information collected

cola_graph - generate a graph based on all the information collected where nodes do not overlap (it might take a while to generate the graph if there are lots of nodes)

gclean - clear graph information

show - show all information that will be collected

run - extract intelligece information

help - this help

plugins - show available plugins

执行下面的命令收集指定的IoCs的相关信息：

>md5=747b3fd525de1af0a56985aa29779b86,2fdeb22d2fa29878dca12fb493df24df

>domain=tinyor.info

>ip=195.22.26.248

>email=jgou.veia@gmail.com

>asn=16276

>run

Output created in C:\Users\Roberto\Documents\GitHub\OSTrICa\report\a0b983ae-e30a-46dc-a1d0-b59e661595c0

> graph

Graph generated in C:\Users\Roberto\Documents\GitHub\OSTrICa\viz\f4da8f02-ec9c-4700-9345-bd715de7789f.html

要想输出详细信息，可以启用cfg.py文件中的DEBUG选项。输出会有些混杂，但是会显示更加详细的信息，例子如下：

> run

Running DeepViz() on 747b3fd525de1af0a56985aa29779b86

Running VT() on 747b3fd525de1af0a56985aa29779b86

cleanup VirusTotal...

Running DeepViz() on 2fdeb22d2fa29878dca12fb493df24df

Running VT() on 2fdeb22d2fa29878dca12fb493df24df

cleanup VirusTotal...

Running BlackListChecker() on tinyor.info

cleanup BlackListChecker...

Running DomainBigData() on tinyor.info

cleanup DomainBigData...

以下2个命令可以生成图像

graph, 基于所有收集的信息生成图像

cola_graph, 基于所有收集的信息生成图像，并且不会重叠节点。

当前可用的插件

下面的列表列出了当前可用的插件：

ThreatCrowd – 开发者 Ptr32Void

BlackLists – 开发者 Ptr32Void

CymruWhois – 开发者 Ptr32Void

DomainBigData – 开发者 Ptr32Void

NortonSafeWeb – 开发者 Ptr32Void

PyWhois – 开发者 Ptr32Void

SafeBrowsing – 开发者 Ptr32Void

SpyOnWeb – 开发者 Ptr32Void

TCPIPutils – 开发者 Ptr32Void

VirusTotal – 开发者 Ptr32Void

WebSiteInformer – 开发者 Ptr32Void

WhoisXmlApi – 开发者 Ptr32Void

如何开发新的插件

插件都保存在Plugins文件夹下。要想创建新的插件，你需要在Plugins文件夹下创建新的子文件夹，并在里面添加一个新的__init__.py。

OSTriCa会调用每个插件里的两个函数run和data_visualization,定义如下：

# intelligence is the IoC provided (eg.: something@yahoo.com)

# extraction_type is the typology (eg.: an MD5 or email, etc)

def run(intelligence, extraction_type):

# function run is the core part of the plugin. It is used to collect the information and afterwards it returns back JSON data as per below:

.... code used to collect Intelligence ....

# a dictionary where extraction_type is the type (md5, email, etc) and intelligence_dictionary is the JSON data collected by the plugin

return {'extraction_type': extraction_type, 'intelligence_information':intelligence_dictionary}

# nodes are passed by OSTrICa itself and should never be overwritten but updated because they might contain details related to the previously collected information

# edges are passed by OSTrICa itself and should never be overwritten but updated because they might contain details related to the previously collected information

# json_data is the json output collected by the plugin

def data_visualization(nodes, edges, json_data):

data_visualization需要返回nodes和edges，OSTrICa会用到它们。如果没有要显示的数据，可以用下面的代码：

def data_visualization(nodes, edges, json_data):

return nodes, edges

你需要在文件的上方添加如下的导入和变量。

from ostrica.utilities.cfg import Config as cfg # used to include configuration data

# used to identify what kind of data the plugin can extract:

# ip = IP Address information

# domain = Domain information

# asn = ASN information

# md5 = MD5 information

# sha256 = SHA256 information

# email = Email information

extraction_type = [cfg.intelligence_type['ip'], cfg.intelligence_type['domain'], cfg.intelligence_type['asn']]

# True if plugin is enabled, False if not

enabled = True

# Plugin Version

version = 0.1

# Developer(s) name and contact

developer = 'Your Name '

# Plugin Description

description = 'Plugin used to collect information about IPs, domains or ASNs on SafeBrowsing'

# True if visualization module is available for the plugin, False otherwise

visual_data = True