Skip to content

anl-cyberscience/FlexTransform

BranchesTags

Repository files navigation

Flexible Transform

Build Status PyPI version PyPI

Flexible Transform (FlexT) enables dynamic translation between formats, accomplishing this by digesting CTI data down to its semantic roots (meaning and context).

Install

FlexT requires Python3 & is available via pip, but it requires the python package lxml which has unix dependencies such as libxml2 and libxslt (as well as associated development packages). For systems that employ apt-get such as Debian & Ubuntu, the following command can be used.

$ sudo apt-get install libxml2-dev libxslt-dev python-dev

pip command:

$ pip install FlexTransform

Usage

Currently, FlexT supports Command-Line access as well as functioning as a Python Library, while future development will add a RESTful API with a local web server.

Python Library

FlexT accepts File-like objects, so in addition to allowing for the open command, you can also use python objects like StringIO.

from FlexTransform import FlexTransform
flexT = FlexTransform.FlexTransform()

with open("/Users/cfm/FlexT/FlexTransform/resources/sampleConfigurations/cfm13.cfg", "r") as input_cfg:
        flexT.AddParser("cfm13", input_cfg)
with open("/Users/cfm/FlexT/FlexTransform/resources/sampleConfigurations/stix_tlp.cfg", "r") as output_cfg:
        flexT.AddParser("stix", output_cfg)

with open("/Users/cfm/input.xml", "r") as input_file:
        with open("/Users/cfm/output.xml", "w") as output_file:
                flexT.TransformFile(input_file, "cfm13", "stix", targetFileName=output_file)

Command Line

$ flext --src inputFile.txt --src-config srcConfig.cfg --dst outputFile.xml --dst-config dstConfig.cfg
  • Required arguments
    • src - Source file
    • src-config - Source file parser configuration
    • dst - Destination file
    • dst-config - Destination file parser configuration
  • Optional arguments
    • src-metadata - Source metadata file
    • tbox-uri - The rui location of the tbox file
    • source-schema-IRI - Ontological IRI for the source
    • destination-schema-IRI - Ontological IRI for the destination

Example - Command Line

  • Input Files (present in the repo, not the package, download to known location if needed)
    • [CFM13 File](./Example Files/SampleInput-CFM13.xml)
    • [STIX-TLP File](./Example Files/SampleInput-STIX-TLP.xml)
  • Command

    • CFM13 to STIX-TLP

      flext --src FlexTransform/ExampleFiles/SampleInput-CFM13.xml --src-config FlexTransform/FlexTransform/resources/sampleConfigurations/cfm13.cfg --dst Output-STIX-TLP.xml --dst-config FlexTransform/FlexTransform/resources/sampleConfigurations/stix_tlp.cfg

    • STIX-TLP to CFM13

      flext --src FlexTransform/ExampleFiles/SampleInput-STIX-TLP.xml --src-config FlexTransform/FlexTransform/resources/sampleConfigurations/stix_tlp.cfg --dst Output-CFM13.xml --dst-config FlexTransform/FlexTransform/resources/sampleConfigurations/cfm13.cfg
  • Output

    • CFM13 to STIX-TLP

      <stix:STIX_Package 
	xmlns:cyboxCommon="http://cybox.mitre.org/common-2"
	xmlns:cybox="http://cybox.mitre.org/cybox-2"
	xmlns:cyboxVocabs="http://cybox.mitre.org/default_vocabularies-2"
	xmlns:AddressObj="http://cybox.mitre.org/objects#AddressObject-2"
	xmlns:PortObj="http://cybox.mitre.org/objects#PortObject-2"
	xmlns:marking="http://data-marking.mitre.org/Marking-1"
	xmlns:tlpMarking="http://data-marking.mitre.org/extensions/MarkingStructure#TLP-1"
	xmlns:coa="http://stix.mitre.org/CourseOfAction-1"
	xmlns:indicator="http://stix.mitre.org/Indicator-2"
	xmlns:stixCommon="http://stix.mitre.org/common-1"
	xmlns:stixVocabs="http://stix.mitre.org/default_vocabularies-1"
	xmlns:stix="http://stix.mitre.org/stix-1"
	xmlns:CFM="http://www.anl.gov/cfm/stix"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="
	http://cybox.mitre.org/common-2 http://cybox.mitre.org/XMLSchema/common/2.1/cybox_common.xsd
	http://cybox.mitre.org/cybox-2 http://cybox.mitre.org/XMLSchema/core/2.1/cybox_core.xsd
	http://cybox.mitre.org/default_vocabularies-2 http://cybox.mitre.org/XMLSchema/default_vocabularies/2.1/cybox_default_vocabularies.xsd
	http://cybox.mitre.org/objects#AddressObject-2 http://cybox.mitre.org/XMLSchema/objects/Address/2.1/Address_Object.xsd
	http://cybox.mitre.org/objects#PortObject-2 http://cybox.mitre.org/XMLSchema/objects/Port/2.1/Port_Object.xsd
	http://data-marking.mitre.org/Marking-1 http://stix.mitre.org/XMLSchema/data_marking/1.1.1/data_marking.xsd
	http://data-marking.mitre.org/extensions/MarkingStructure#TLP-1 http://stix.mitre.org/XMLSchema/extensions/marking/tlp/1.1.1/tlp_marking.xsd
	http://stix.mitre.org/CourseOfAction-1 http://stix.mitre.org/XMLSchema/course_of_action/1.1.1/course_of_action.xsd
	http://stix.mitre.org/Indicator-2 http://stix.mitre.org/XMLSchema/indicator/2.1.1/indicator.xsd
	http://stix.mitre.org/common-1 http://stix.mitre.org/XMLSchema/common/1.1.1/stix_common.xsd
	http://stix.mitre.org/default_vocabularies-1 http://stix.mitre.org/XMLSchema/default_vocabularies/1.1.1/stix_default_vocabularies.xsd
	http://stix.mitre.org/stix-1 http://stix.mitre.org/XMLSchema/core/1.1.1/stix_core.xsd" id="CFM:STIXPackage-722cede7-e98e-53db-b3a9-192a0c6166cb" version="1.1.1" timestamp="2016-05-20T20:43:24+00:00">
  <stix:STIX_Header>
      <stix:Package_Intent xsi:type="stixVocabs:PackageIntentVocab-1.0">Indicators</stix:Package_Intent>
      <stix:Handling>
          <marking:Marking>
              <marking:Controlled_Structure>//node() | //@*</marking:Controlled_Structure>
              <marking:Marking_Structure xsi:type='tlpMarking:TLPMarkingStructureType' color="AMBER"/>
          </marking:Marking>
      </stix:Handling>
      <stix:Information_Source>
          <stixCommon:Description>Fake National Lab</stixCommon:Description>
          <stixCommon:Identity>
              <stixCommon:Name>Fake</stixCommon:Name>
          </stixCommon:Identity>
          <stixCommon:Time>
              <cyboxCommon:Produced_Time>2016-02-21T22:50:02+06:00</cyboxCommon:Produced_Time>
          </stixCommon:Time>
      </stix:Information_Source>
  </stix:STIX_Header>
  <stix:Indicators>
      <stix:Indicator id="CFM:Indicator-2b2d04ff-b597-5f30-bd6e-e7741e91d1ed" timestamp="2016-05-20T20:43:24+00:00" xsi:type='indicator:IndicatorType' version="2.1.1">
          <indicator:Type xsi:type="stixVocabs:IndicatorTypeVocab-1.1">IP Watchlist</indicator:Type>
          <indicator:Description>SSH scans against multiple hosts, direction:ingress, confidence:87, severity:high</indicator:Description>
          <indicator:Observable id="CFM:Observable-44b81e1b-f77b-5903-b4a7-5c56c9c5748b" sighting_count="1">
              <cybox:Keywords>
                  <cybox:Keyword>Scanning</cybox:Keyword>
              </cybox:Keywords>
              <cybox:Object id="CFM:Object-da05a4ba-1626-57c8-9a7b-bcaf514c43e7">
                  <cybox:Properties xsi:type="AddressObj:AddressObjectType" category="ipv4-addr">
                      <AddressObj:Address_Value condition="Equals">10.10.10.10</AddressObj:Address_Value>
                  </cybox:Properties>
                  <cybox:Related_Objects>
                      <cybox:Related_Object id="CFM:Object-7ca69e67-d908-55da-8a42-2e0d4cf8fbaf">
                          <cybox:Properties xsi:type="PortObj:PortObjectType">
                              <PortObj:Port_Value>22</PortObj:Port_Value>
                              <PortObj:Layer4_Protocol>TCP</PortObj:Layer4_Protocol>
                          </cybox:Properties>
                          <cybox:Relationship xsi:type="cyboxVocabs:ObjectRelationshipVocab-1.1">Connected_To</cybox:Relationship>
                      </cybox:Related_Object>
                  </cybox:Related_Objects>
              </cybox:Object>
          </indicator:Observable>
          <indicator:Suggested_COAs>
              <indicator:Suggested_COA>
                  <stixCommon:Course_Of_Action id="CFM:COA-7a9ed7c3-4872-51cc-83e4-3f0600cc400d" xsi:type='coa:CourseOfActionType'>
                      <coa:Stage>Remedy</coa:Stage>
                      <coa:Type>Perimeter Blocking</coa:Type>
                  </stixCommon:Course_Of_Action>
              </indicator:Suggested_COA>
          </indicator:Suggested_COAs>
          <indicator:Sightings sightings_count="12">
              <indicator:Sighting timestamp="2016-02-21T22:45:53-04:00" timestamp_precision="second"/>
          </indicator:Sightings>
      </stix:Indicator>
  </stix:Indicators>
</stix:STIX_Package>
      • STIX-TLP to CFM13
      <?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE IDMEF-Message PUBLIC "-//IETF//DTD RFC XXXX IDMEF v1.0//EN" "idmef-message.dtd">
<IDMEF-Message xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.anl.gov/cfm/1.3/IDMEF-Message" xsi:schemaLocation="http://www.anl.gov/cfm/1.3/IDMEF-Message/../../../resources/schemas/CFMMessage13.xsd">
  <Alert>
    <Analyzer analyzerid="Fake">
      <Node>
        <location>1325 G St, NW, Suite 600, Washington DC 20005</location>
        <name>Operations Desk, 404-446-9780, [email protected]</name>
      </Node>
    </Analyzer>
    <AnalyzerTime>2016-03-23T16:45:05+0400</AnalyzerTime>
    <AdditionalData type="string" meaning="report schedule">NoValue</AdditionalData>
    <AdditionalData type="integer" meaning="number of alerts in this report">2</AdditionalData>
    <AdditionalData type="string" meaning="report type">alerts</AdditionalData>
    <AdditionalData type="date-time" meaning="report start time">2016-03-23T16:45:05+0400</AdditionalData>
  </Alert>
  <Alert>
    <CreateTime>2016-03-23T16:45:05+0400</CreateTime>
    <Source>
      <Node>
        <Address category="ipv4-addr">
          <address>10.10.10.10</address>
        </Address>
      </Node>
    </Source>
    <Classification text="CRISP Report Indicator">
      <Reference meaning="Unspecified" origin="user-specific">
        <name>unknown</name>
        <url> </url>
      </Reference>
    </Classification>
    <Assessment>
      <Action category="block-installed"/>
    </Assessment>
    <AdditionalData type="integer" meaning="recon">0</AdditionalData>
    <AdditionalData type="integer" meaning="OUO">0</AdditionalData>
    <AdditionalData type="integer" meaning="duration">86400</AdditionalData>
    <AdditionalData type="string" meaning="restriction">public</AdditionalData>
  </Alert>
  <Alert>
    <CreateTime>2016-03-23T16:45:05+0400</CreateTime>
    <Source>
      <Node>
        <Address>
          <address>bad.domain.be/poor/path</address>
        </Address>
      </Node>
    </Source>
    <Classification text="URL Block: CRISP Report Indicator">
      <Reference meaning="Unspecified" origin="user-specific">
        <name>unknown</name>
        <url> </url>
      </Reference>
    </Classification>
    <Assessment>
      <Action category="block-installed"/>
    </Assessment>
    <AdditionalData type="integer" meaning="recon">0</AdditionalData>
    <AdditionalData type="integer" meaning="OUO">0</AdditionalData>
    <AdditionalData type="integer" meaning="duration">86400</AdditionalData>
    <AdditionalData type="string" meaning="restriction">public</AdditionalData>
  </Alert>
</IDMEF-Message>

Contributing

Bug Reports & Feature Requests

Please use the issue tracker to report any bugs or file feature requests.

Developing

Additional functionality is always being added, but we welcome any PRs to improve the project.

About

Enables dynamic translation of structured data between formats

cfm.gss.anl.gov/flexible-transform/

Topics

python python-library stix

Resources

Readme

License

View license
Activity
Custom properties

Stars

14 stars

Watchers

15 watching

Forks

2 forks
Report repository

Releases

5 tags

Packages

No packages published

Contributors 11

Languages