Removed urllib3 dependency (#10)

* Removed urllib3 dependency * Update README.md * Update release notes * Revert min_phantom_version and resolving static job failures * Update README.md * Resolving lint job * Removed docutils and update release notes * Update license year in LICENSE file --------- Co-authored-by: splunk-soar-connectors-admin <admin@splunksoar>
splunk-soar-connectors · Apr 2, 2024 · 3160f4c · 3160f4c
1 parent 223b943
commit 3160f4c
Show file tree

Hide file tree

Showing 24 changed files with 84 additions and 76 deletions.
diff --git a/.github/workflows/linting.yml b/.github/workflows/linting.yml
@@ -1,7 +1,7 @@
 name: Linting
 on: [push, pull_request]
 jobs:
-  lint: 
+  lint:
     # Run per push for internal contributers. This isn't possible for forked pull requests,
     # so we'll need to run on PR events for external contributers.
     # String comparison below is case insensitive.

diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -1,6 +1,6 @@
 repos:
 -   repo: https://github.com/phantomcyber/dev-cicd-tools
-    rev: v1.16
+    rev: v1.17
     hooks:
     -   id: org-hook
     -   id: package-app-dependencies

diff --git a/LICENSE b/LICENSE
@@ -186,7 +186,7 @@
       same "printed page" as the copyright notice for easier
       identification within third-party archives.
 
-   Copyright 2021 Splunk Inc.
+   Copyright (c) 2017-2024 Splunk Inc.
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
@@ -198,4 +198,4 @@
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
-   limitations under the License.
+   limitations under the License.
diff --git a/NOTICE b/NOTICE
@@ -1,5 +1,5 @@
 Splunk SOAR AWS Athena
-Copyright (c) 2017-2021 Splunk Inc.
+Copyright (c) 2017-2024 Splunk Inc.
 
 Third-party Software Attributions:
 

diff --git a/README.md b/README.md
@@ -2,16 +2,16 @@
 # AWS Athena
 
 Publisher: Splunk  
-Connector Version: 2\.2\.6  
+Connector Version: 2.2.8  
 Product Vendor: AWS  
 Product Name: Athena  
-Product Version Supported (regex): "\.\*"  
-Minimum Product Version: 4\.9\.39220  
+Product Version Supported (regex): ".\*"  
+Minimum Product Version: 4.9.39220  
 
 This app supports investigative actions on AWS Athena
 
-[comment]: # " File: readme.md"
-[comment]: # "  Copyright (c) 2017-2021 Splunk Inc."
+[comment]: # " File: README.md"
+[comment]: # "  Copyright (c) 2017-2024 Splunk Inc."
 [comment]: # ""
 [comment]: # "  Licensed under Apache 2.0 (https://www.apache.org/licenses/LICENSE-2.0.txt)"
 [comment]: # ""
@@ -47,11 +47,11 @@ The below configuration variables are required for this Connector to operate.  T
 
 VARIABLE | REQUIRED | TYPE | DESCRIPTION
 -------- | -------- | ---- | -----------
-**access\_key** |  optional  | password | AWS Access Key
-**secret\_key** |  optional  | password | AWS Secret Key
+**access_key** |  optional  | password | AWS Access Key
+**secret_key** |  optional  | password | AWS Secret Key
 **region** |  required  | string | Default Region
-**kms\_key** |  optional  | password | KMS key
-**use\_role** |  optional  | boolean | Use attached role when running Phantom in EC2
+**kms_key** |  optional  | password | KMS key
+**use_role** |  optional  | boolean | Use attached role when running Phantom in EC2
 
 ### Supported Actions  
 [test connectivity](#action-test-connectivity) - Validate the asset configuration for connectivity using supplied configuration  
@@ -82,56 +82,62 @@ PARAMETER | REQUIRED | DESCRIPTION | TYPE | CONTAINS
 **credentials** |  optional  | Assumed role credentials | string |  `aws credentials` 
 
 #### Action Output
-DATA PATH | TYPE | CONTAINS
---------- | ---- | --------
-action\_result\.status | string | 
-action\_result\.data\.\*\.NamedQuery\.Database | string | 
-action\_result\.data\.\*\.NamedQuery\.Description | string | 
-action\_result\.data\.\*\.NamedQuery\.Name | string | 
-action\_result\.data\.\*\.NamedQuery\.NamedQueryId | string |  `athena named query` 
-action\_result\.data\.\*\.NamedQuery\.QueryString | string |  `athena query` 
-action\_result\.data\.\*\.ResponseMetadata\.HTTPHeaders\.connection | string | 
-action\_result\.data\.\*\.ResponseMetadata\.HTTPHeaders\.content\-length | string | 
-action\_result\.data\.\*\.ResponseMetadata\.HTTPHeaders\.content\-type | string | 
-action\_result\.data\.\*\.ResponseMetadata\.HTTPHeaders\.date | string | 
-action\_result\.data\.\*\.ResponseMetadata\.HTTPHeaders\.x\-amzn\-requestid | string | 
-action\_result\.data\.\*\.ResponseMetadata\.HTTPStatusCode | numeric | 
-action\_result\.data\.\*\.ResponseMetadata\.RequestId | string | 
-action\_result\.data\.\*\.ResponseMetadata\.RetryAttempts | numeric | 
-action\_result\.summary\.num\_queries | numeric | 
-action\_result\.message | string | 
-summary\.total\_objects | numeric | 
-summary\.total\_objects\_successful | numeric | 
-action\_result\.parameter\.credentials | string |  `aws credentials`   
+DATA PATH | TYPE | CONTAINS | EXAMPLE VALUES
+--------- | ---- | -------- | --------------
+action_result.status | string |  |   success  failed 
+action_result.data.\*.NamedQuery.Database | string |  |   sampledb 
+action_result.data.\*.NamedQuery.Description | string |  |   Sample query to get the top 10 airports with the most number of departures since 2000 
+action_result.data.\*.NamedQuery.Name | string |  |   Flights Select Query 
+action_result.data.\*.NamedQuery.NamedQueryId | string |  `athena named query`  |   1234abcd-12ab-ab12-ab12-123456abcdef 
+action_result.data.\*.NamedQuery.QueryString | string |  `athena query`  |   SELECT origin, count(\*) AS total_departures
+FROM
+test_table
+WHERE year >= '2000'
+GROUP BY origin
+ORDER BY total_departures DESC
+LIMIT 10; 
+action_result.data.\*.ResponseMetadata.HTTPHeaders.connection | string |  |   keep-alive 
+action_result.data.\*.ResponseMetadata.HTTPHeaders.content-length | string |  |   389 
+action_result.data.\*.ResponseMetadata.HTTPHeaders.content-type | string |  |   application/x-amz-json-1.1 
+action_result.data.\*.ResponseMetadata.HTTPHeaders.date | string |  |   Tue, 03 Oct 2017 23:24:08 GMT  Tue, 03 Oct 2017 23:29:58 GMT 
+action_result.data.\*.ResponseMetadata.HTTPHeaders.x-amzn-requestid | string |  |   1234abcd-12ab-ab12-ab12-123456abcdef 
+action_result.data.\*.ResponseMetadata.HTTPStatusCode | numeric |  |   200 
+action_result.data.\*.ResponseMetadata.RequestId | string |  |   1234abcd-12ab-ab12-ab12-123456abcdef 
+action_result.data.\*.ResponseMetadata.RetryAttempts | numeric |  |   0 
+action_result.summary.num_queries | numeric |  |   7 
+action_result.message | string |  |   Num queries: 7 
+summary.total_objects | numeric |  |   1 
+summary.total_objects_successful | numeric |  |   1 
+action_result.parameter.credentials | string |  `aws credentials`  |   {'AccessKeyId': 'ASIASJL6ZZZZZ3M7QC2J', 'Expiration': '2021-06-07 22:28:04', 'SecretAccessKey': 'ZZZZZAmvLPictcVBPvjJx0d7MRezOuxiLCMZZZZZ', 'SessionToken': 'ZZZZZXIvYXdzEN///////////wEaDFRU0s4AVrw0k0oYICK4ATAzOqzAkg9bHY29lYmP59UvVOHjLufOy4s7SnAzOxGqGIXnukLis4TWNhrJl5R5nYyimrm6K/9d0Cw2SW9gO0ZRjEJHWJ+yY5Qk2QpWctS2BGn4n+G8cD6zEweCCMj+ScI5p8n7YI4wOdvXvOsVMmjV6F09Ujqr1w+NwoKXlglznXGs/7Q1kNZOMiioEhGUyoiHbQb37GCKslDK+oqe0KNaUKQ96YCepaLgMbMquDgdAM8I0TTxUO0o5ILF/gUyLT04R7QlOfktkdh6Qt0atTS+xeKi1hirKRizpJ8jjnxGQIikPRToL2v3ZZZZZZ=='}   
 
 ## action: 'run query'
 Run a named query on Athena
 
 Type: **investigate**  
 Read only: **True**
 
-The <b>query</b> parameter can take either a named query ID, or a query string\.<br><br>If the <b>database</b> parameter is not included, the query will run on the default database configured on Athena\.<br><br>The AWS API requires Athena query results be stored in a location on S3\. Use the <b>s3\_location</b> parameter to specify this location\.<br><br>To encypt the files containing the results, specify the desired encryption scheme with the <b>encryption</b> parameter\. If the given encryption scheme requires a KMS key, the action will use the <b>kms\_key</b> app configuration parameter\.
+The <b>query</b> parameter can take either a named query ID, or a query string.<br><br>If the <b>database</b> parameter is not included, the query will run on the default database configured on Athena.<br><br>The AWS API requires Athena query results be stored in a location on S3. Use the <b>s3_location</b> parameter to specify this location.<br><br>To encypt the files containing the results, specify the desired encryption scheme with the <b>encryption</b> parameter. If the given encryption scheme requires a KMS key, the action will use the <b>kms_key</b> app configuration parameter.
 
 #### Action Parameters
 PARAMETER | REQUIRED | DESCRIPTION | TYPE | CONTAINS
 --------- | -------- | ----------- | ---- | --------
 **query** |  required  | Query to run | string |  `athena query`  `athena named query` 
 **database** |  optional  | Database to run query on | string | 
-**s3\_location** |  required  | S3 location to save results | string | 
+**s3_location** |  required  | S3 location to save results | string | 
 **encryption** |  optional  | Encyption scheme of S3 location | string | 
 **credentials** |  optional  | Assumed role credentials | string |  `aws credentials` 
 
 #### Action Output
-DATA PATH | TYPE | CONTAINS
---------- | ---- | --------
-action\_result\.status | string | 
-action\_result\.parameter\.database | string | 
-action\_result\.parameter\.encryption | string | 
-action\_result\.parameter\.query | string |  `athena query`  `athena named query` 
-action\_result\.parameter\.s3\_location | string | 
-action\_result\.data\.\*\.\*\.VarCharValue | string | 
-action\_result\.summary\.num\_rows | numeric | 
-action\_result\.message | string | 
-summary\.total\_objects | numeric | 
-summary\.total\_objects\_successful | numeric | 
-action\_result\.parameter\.credentials | string |  `aws credentials` 
+DATA PATH | TYPE | CONTAINS | EXAMPLE VALUES
+--------- | ---- | -------- | --------------
+action_result.status | string |  |   success  failed 
+action_result.parameter.database | string |  |   sampledb 
+action_result.parameter.encryption | string |  |   SSE_S3 
+action_result.parameter.query | string |  `athena query`  `athena named query`  |   1234abcd-12ab-ab12-ab12-123456abcdef 
+action_result.parameter.s3_location | string |  |   s3://test-bucket/test-queries  s3://test-bucket/test-queries/kms_encrypt 
+action_result.data.\*.\*.VarCharValue | string |  |   os 
+action_result.summary.num_rows | numeric |  |   0 
+action_result.message | string |  |   Num rows: 0 
+summary.total_objects | numeric |  |   1 
+summary.total_objects_successful | numeric |  |   1 
+action_result.parameter.credentials | string |  `aws credentials`  |   {'AccessKeyId': 'ASIASJL6ZZZZZ3M7QC2J', 'Expiration': '2021-06-07 22:28:04', 'SecretAccessKey': 'ZZZZZAmvLPictcVBPvjJx0d7MRezOuxiLCMZZZZZ', 'SessionToken': 'ZZZZZXIvYXdzEN///////////wEaDFRU0s4AVrw0k0oYICK4ATAzOqzAkg9bHY29lYmP59UvVOHjLufOy4s7SnAzOxGqGIXnukLis4TWNhrJl5R5nYyimrm6K/9d0Cw2SW9gO0ZRjEJHWJ+yY5Qk2QpWctS2BGn4n+G8cD6zEweCCMj+ScI5p8n7YI4wOdvXvOsVMmjV6F09Ujqr1w+NwoKXlglznXGs/7Q1kNZOMiioEhGUyoiHbQb37GCKslDK+oqe0KNaUKQ96YCepaLgMbMquDgdAM8I0TTxUO0o5ILF/gUyLT04R7QlOfktkdh6Qt0atTS+xeKi1hirKRizpJ8jjnxGQIikPRToL2v3ZZZZZZ=='} 
diff --git a/__init__.py b/__init__.py
@@ -1,6 +1,6 @@
 # File: __init__.py
 #
-# Copyright (c) 2017-2021 Splunk Inc.
+# Copyright (c) 2017-2024 Splunk Inc.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

diff --git a/athena.json b/athena.json
@@ -9,8 +9,8 @@
     "product_name": "Athena",
     "product_version_regex": ".*",
     "publisher": "Splunk",
-    "license": "Copyright (c) 2017-2021 Splunk Inc.",
-    "app_version": "2.2.7",
+    "license": "Copyright (c) 2017-2024 Splunk Inc.",
+    "app_version": "2.2.8",
     "utctime_updated": "2022-01-07T22:02:43.000000Z",
     "package_name": "phantom_athena",
     "main_module": "athena_connector.py",
@@ -25,35 +25,27 @@
         "wheel": [
             {
                 "module": "boto3",
-                "input_file": "wheels/boto3-1.17.30-py2.py3-none-any.whl"
+                "input_file": "wheels/shared/boto3-1.17.30-py2.py3-none-any.whl"
             },
             {
                 "module": "botocore",
-                "input_file": "wheels/botocore-1.20.30-py2.py3-none-any.whl"
-            },
-            {
-                "module": "docutils",
-                "input_file": "wheels/docutils-0.16-py2.py3-none-any.whl"
+                "input_file": "wheels/shared/botocore-1.20.30-py2.py3-none-any.whl"
             },
             {
                 "module": "jmespath",
-                "input_file": "wheels/jmespath-0.10.0-py2.py3-none-any.whl"
+                "input_file": "wheels/shared/jmespath-0.10.0-py2.py3-none-any.whl"
             },
             {
                 "module": "python_dateutil",
-                "input_file": "wheels/python_dateutil-2.8.2-py2.py3-none-any.whl"
+                "input_file": "wheels/shared/python_dateutil-2.9.0.post0-py2.py3-none-any.whl"
             },
             {
                 "module": "s3transfer",
-                "input_file": "wheels/s3transfer-0.3.7-py2.py3-none-any.whl"
+                "input_file": "wheels/shared/s3transfer-0.3.7-py2.py3-none-any.whl"
             },
             {
                 "module": "six",
-                "input_file": "wheels/six-1.16.0-py2.py3-none-any.whl"
-            },
-            {
-                "module": "urllib3",
-                "input_file": "wheels/urllib3-1.26.7-py2.py3-none-any.whl"
+                "input_file": "wheels/shared/six-1.16.0-py2.py3-none-any.whl"
             }
         ]
     },
@@ -434,4 +426,4 @@
             "versions": "EQ(*)"
         }
     ]
-}
+}
diff --git a/athena_connector.py b/athena_connector.py
@@ -1,6 +1,6 @@
 # File: athena_connector.py
 #
-# Copyright (c) 2017-2021 Splunk Inc.
+# Copyright (c) 2017-2024 Splunk Inc.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -179,10 +179,12 @@ def _handle_test_connectivity(self, param):
     def _handle_list_queries(self, param):
 
         action_result = self.add_action_result(ActionResult(dict(param)))
+        self.save_progress("In action handler for: {0}".format(self.get_action_identifier()))
 
         if not self._create_client(action_result, param):
             return action_result.get_status()
 
+        self.debug_print("Making boto3 call to get list of named queries")
         ret_val, resp_json = self._make_boto_call(action_result, 'list_named_queries')
         if (phantom.is_fail(ret_val)):
             return ret_val
@@ -236,6 +238,7 @@ def _handle_run_query(self, param):
 
         if reg_exp.match(query.lower()):
 
+            self.debug_print("Making boto3 call to get named query")
             ret_val, query_json = self._make_boto_call(action_result, 'get_named_query', NamedQueryId=query)
             if (phantom.is_fail(ret_val)):
                 return ret_val
@@ -327,7 +330,7 @@ def handle_action(self, param):
 
     if (len(sys.argv) < 2):
         print("No test json specified as input")
-        exit(0)
+        sys.exit(0)
 
     with open(sys.argv[1]) as f:
         in_json = f.read()
@@ -339,4 +342,4 @@ def handle_action(self, param):
         ret_val = connector._handle_action(json.dumps(in_json), None)
         print(json.dumps(json.loads(ret_val), indent=4))
 
-    exit(0)
+    sys.exit(0)
diff --git a/athena_consts.py b/athena_consts.py
@@ -1,6 +1,6 @@
 # File: athena_consts.py
 #
-# Copyright (c) 2017-2021 Splunk Inc.
+# Copyright (c) 2017-2024 Splunk Inc.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

diff --git a/athena_view.py b/athena_view.py
@@ -1,6 +1,6 @@
 # File: athena_view.py
 #
-# Copyright (c) 2017-2021 Splunk Inc.
+# Copyright (c) 2017-2024 Splunk Inc.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

diff --git a/manual_readme_content.md b/manual_readme_content.md
@@ -1,5 +1,5 @@
 [comment]: # " File: README.md"
-[comment]: # "  Copyright (c) 2017-2021 Splunk Inc."
+[comment]: # "  Copyright (c) 2017-2024 Splunk Inc."
 [comment]: # ""
 [comment]: # "  Licensed under Apache 2.0 (https://www.apache.org/licenses/LICENSE-2.0.txt)"
 [comment]: # ""

diff --git a/release_notes/unreleased.md b/release_notes/unreleased.md
@@ -1 +1,2 @@
 **Unreleased**
+* Removed urllib3 dependency in order to use platform packages [PAPP-33451]
diff --git a/requirements.txt b/requirements.txt
@@ -1,3 +1,2 @@
 boto3==1.17.30
 botocore==1.20.30
-docutils==0.16
diff --git a/run_query.html b/run_query.html
@@ -1,5 +1,5 @@
 <!-- File: run_query.html
-     Copyright (c) 2017-2021 Splunk Inc.
+     Copyright (c) 2017-2024 Splunk Inc.
      Licensed under the Apache License, Version 2.0 (the "License");
      you may not use this file except in compliance with the License.
      You may obtain a copy of the License at

diff --git a/tox.ini b/tox.ini
@@ -0,0 +1,7 @@
+[flake8]
+max-line-length = 145
+max-complexity = 28
+extend-ignore = F403,E128,E126,E111,E121,E127,E731,E201,E202,F405,E722,D,W292
+
+[isort]
+line_length = 145
diff --git a/wheels/docutils-0.16-py2.py3-none-any.whl b/wheels/docutils-0.16-py2.py3-none-any.whl
diff --git a/wheels/python_dateutil-2.8.2-py2.py3-none-any.whl b/wheels/python_dateutil-2.8.2-py2.py3-none-any.whl
diff --git a/wheels/boto3-1.17.30-py2.py3-none-any.whl → ...shared/boto3-1.17.30-py2.py3-none-any.whl b/wheels/boto3-1.17.30-py2.py3-none-any.whl → ...shared/boto3-1.17.30-py2.py3-none-any.whl
diff --git a/wheels/botocore-1.20.30-py2.py3-none-any.whl → ...red/botocore-1.20.30-py2.py3-none-any.whl b/wheels/botocore-1.20.30-py2.py3-none-any.whl → ...red/botocore-1.20.30-py2.py3-none-any.whl
diff --git a/wheels/jmespath-0.10.0-py2.py3-none-any.whl → ...ared/jmespath-0.10.0-py2.py3-none-any.whl b/wheels/jmespath-0.10.0-py2.py3-none-any.whl → ...ared/jmespath-0.10.0-py2.py3-none-any.whl
diff --git a/wheels/shared/python_dateutil-2.9.0.post0-py2.py3-none-any.whl b/wheels/shared/python_dateutil-2.9.0.post0-py2.py3-none-any.whl
diff --git a/wheels/s3transfer-0.3.7-py2.py3-none-any.whl → ...red/s3transfer-0.3.7-py2.py3-none-any.whl b/wheels/s3transfer-0.3.7-py2.py3-none-any.whl → ...red/s3transfer-0.3.7-py2.py3-none-any.whl
diff --git a/wheels/six-1.16.0-py2.py3-none-any.whl → ...ls/shared/six-1.16.0-py2.py3-none-any.whl b/wheels/six-1.16.0-py2.py3-none-any.whl → ...ls/shared/six-1.16.0-py2.py3-none-any.whl
diff --git a/wheels/urllib3-1.26.7-py2.py3-none-any.whl b/wheels/urllib3-1.26.7-py2.py3-none-any.whl
Original file line number	Diff line number	Diff line change
		@@ -1 +1,2 @@
		Unreleased
		* Removed urllib3 dependency in order to use platform packages [PAPP-33451]