Skip to content

Commit

Permalink
Merge pull request #4 from KevKibe/update-readme
Browse files Browse the repository at this point in the history
update tests
  • Loading branch information
KevKibe authored Oct 30, 2024
2 parents 5f716a9 + fe7cf77 commit 5fdfb5a
Show file tree
Hide file tree
Showing 7 changed files with 79 additions and 80 deletions.
9 changes: 5 additions & 4 deletions .github/workflows/test-kaggle-action.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Test Kaggle Action
name: Test Kaggle Script Action

on: [pull_request]

Expand All @@ -10,13 +10,14 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v3

- name: Run Kaggle Action
- name: Test Kaggle Script Action
uses: ./
with:
username: ${{ secrets.KAGGLE_USERNAME }}
key: ${{ secrets.KAGGLE_KEY }}
title: "Kernel Test v2"
custom_script: "pytest tests/test.py"
title: "Test Kaggle Script Action"
custom_script: "python test.py --url "https://jsonplaceholder.typicode.com/posts/1""
working_subdir: "tests"
enable_internet: true
enable_gpu: false
enable_tpu: false
Expand Down
45 changes: 39 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
### Kaggle Script Runner
### Kaggle Script Action

**Kaggle Script Runner** is a GitHub Action designed for automating the execution of custom scripts on Kaggle kernels, making it ideal for continuous integration (CI) and deployment (CD) of machine learning workflows directly within your GitHub repository.
**Kaggle Script Action** is a GitHub Action designed for automating the execution of custom scripts on Kaggle kernels, making it ideal for continuous integration (CI) and deployment (CD) of machine learning workflows directly within your GitHub repository.
<br>
This action pulls the repository on the branch it is executed from, installs dependencies from the `requirements.txt` file in the root directory by default, and supports executing Python scripts on Kaggle's infrastructure with configurable options, including GPU/TPU support, internet access, and custom dataset or kernel dependencies.

Expand All @@ -11,6 +11,17 @@ This action pulls the repository on the branch it is executed from, installs dep
- **CI/CD Integration**: Incorporate machine learning workflows seamlessly into your CI/CD pipeline, with GitHub Actions monitoring the kernel execution status and fetching output logs.
- **Detailed Logging**: Captures and logs the complete output, including kernel execution logs and errors, and displays `data` fields for streamlined debugging.

#### Usage
Sign up for a [Kaggle Account](https://www.kaggle.com/account/login?phase=startRegisterTab) and verify the account.
Go to your account page, then settings. Create your API Token. You'll get a file with something like this:
```json
{
"username": "USERNAME",
"key": "TOKEN"
}
```
Add `USERNAME` and `TOKEN` to Github Actions Secrets as `KAGGLE_USERNAME` and `KAGGLE_KEY`.

#### Inputs
- **`username`** (required): Kaggle username.
- **`key`** (required): Kaggle API token.
Expand All @@ -24,10 +35,32 @@ This action pulls the repository on the branch it is executed from, installs dep
- **`kernel_sources`** (optional): List of other kernel sources formatted as `{username}/{kernel-slug}`.
- **`sleep_time`** (optional): Time interval (in seconds) to wait between kernel status checks. Default: `15`.


#### Example Usage
This GitHub Action fetches logs and kernel output after execution, making it ideal for:
- Running model training or inference scripts in an automated fashion on Kaggle’s infrastructure.
- Integrating Kaggle-based tests for datasets, models, or research pipelines as part of CI/CD workflows.
- Leveraging Kaggle’s GPUs/TPUs for testing model performance and handling large data without local resource constraints.
```yaml
name: Run Tests

on: [pull_request]

jobs:
test_kaggle_action:
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v3

- name: Run Kaggle Action
uses: Kev
with:
username: ${{ secrets.KAGGLE_USERNAME }}
key: ${{ secrets.KAGGLE_KEY }}
title: "Kernel Test v2"
custom_script: "pytest tests/test.py"
enable_internet: true
enable_gpu: false
enable_tpu: false
sleep_time: 10
```
**Note**: To use this action, you must have a Kaggle API token with the appropriate permissions for kernels and datasets.
14 changes: 11 additions & 3 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,10 @@ inputs:
description: Duration (in seconds) before checking the status of kernel execution completion
required: false
default: 15
working_subdir:
description: Subdirectory inside /kaggle/working/$REPO_NAME where dependencies are installed and the custom script is run
required: false
default: ""

outputs:
automatic_releases_tag:
Expand Down Expand Up @@ -110,9 +114,13 @@ runs:
log_message "Cloning repository URL: $REPO_URL on branch: $BRANCH_NAME"
CUSTOM_SCRIPT="${{ inputs.custom_script }}"
log_message "Adding custom script to notebook: $CUSTOM_SCRIPT"
WORKING_SUBDIR="${{ inputs.working_subdir }}"
TARGET_DIR="/kaggle/working/$REPO_NAME/$WORKING_SUBDIR"
dependencies="!cd $TARGET_DIR/$REPO_NAME && pip install -r requirements.txt"
dependencies="!cd /kaggle/working/$REPO_NAME && pip install -r requirements.txt"
log_message "Adding dependency installation script to notebook: $dependencies"
log_message "Adding custom script to notebook: $CUSTOM_SCRIPT"
cat <<EOM > "$OUTPUT_NOTEBOOK"
{
Expand Down Expand Up @@ -141,7 +149,7 @@ runs:
"metadata": {},
"outputs": [],
"source": [
"!cd /kaggle/working/$REPO_NAME && $CUSTOM_SCRIPT"
"!cd $TARGET_DIR && $CUSTOM_SCRIPT"
]
}
],
Expand Down
2 changes: 0 additions & 2 deletions requirements.txt

This file was deleted.

24 changes: 0 additions & 24 deletions tests/fetch_data.py

This file was deleted.

1 change: 1 addition & 0 deletions tests/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
requests
64 changes: 23 additions & 41 deletions tests/test.py
Original file line number Diff line number Diff line change
@@ -1,42 +1,24 @@
import unittest
from unittest.mock import patch
import requests
from fetch_data import fetch_data

class TestFetchData(unittest.TestCase):

@patch('requests.get')
def test_fetch_data_success(self, mock_get):
mock_response = unittest.mock.Mock()
mock_response.raise_for_status.return_value = None # Simulate no HTTP error
mock_response.json.return_value = {"userId": 1, "id": 1, "title": "Test Post", "body": "This is a test."}
mock_get.return_value = mock_response

url = "https://jsonplaceholder.typicode.com/posts/1"
result = fetch_data(url)

self.assertEqual(result, {"userId": 1, "id": 1, "title": "Test Post", "body": "This is a test."})

@patch('requests.get')
def test_fetch_data_http_error(self, mock_get):
mock_response = unittest.mock.Mock()
mock_response.raise_for_status.side_effect = requests.exceptions.HTTPError("404 Not Found")
mock_get.return_value = mock_response

url = "https://jsonplaceholder.typicode.com/posts/1"
result = fetch_data(url)

self.assertIsNone(result)

@patch('requests.get')
def test_fetch_data_request_exception(self, mock_get):
mock_get.side_effect = requests.exceptions.RequestException("Network error")

url = "https://jsonplaceholder.typicode.com/posts/1"
result = fetch_data(url)

self.assertIsNone(result)


if __name__ == '__main__':
unittest.main()
import argparse

def fetch_data(url):
"""
Fetches data from the given URL and returns it as JSON.
"""
try:
response = requests.get(url)
response.raise_for_status()
return response.json()
except requests.exceptions.RequestException as e:
print(f"An error occurred: {e}")
return None

if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Fetch JSON data from a specified URL.")
parser.add_argument('-u', '--url', type=str, required=True, help="The URL to fetch data from")

args = parser.parse_args()
url = args.url

data = fetch_data(url)
print(data)

0 comments on commit 5fdfb5a

Please sign in to comment.