Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Files with a "." in their directory path can't be used as dataset folders #2488

Open
1 task done
terrarier2111 opened this issue Jan 7, 2025 · 3 comments
Open
1 task done

Comments

@terrarier2111
Copy link
Contributor

Describe the bug

This doesn't work and i am told, that there are no normal images detected:

Folder(
name=model_name,
root="/home/user/.config/data",
task=TaskType.CLASSIFICATION,
normal_dir="good",
abnormal_dir="bad",
image_size=(256, 256)
)

This doeswork as expected:

Folder(
name=model_name,
root="/home/user/data",
task=TaskType.CLASSIFICATION,
normal_dir="good",
abnormal_dir="bad",
image_size=(256, 256)
)

Dataset

N/A

Model

PatchCore

Steps to reproduce the behavior

Just try using a folder with a folder inside .config (i suspect this probably works with all folders that have a parent containing a "." in its name)

OS information

OS information:

  • OS: Fedora Linux 40
  • Python Version: 3.11

Expected behavior

The model should just accept my training images

Screenshots

No response

Pip/GitHub

pip

What version/branch did you use?

No response

Configuration YAML

No config

Logs

Traceback (most recent call last):
  File "/home/user/projects/anom/main.py", line 100, in <module>
    train(data_path=args[2], save_path=args[3], model_name=args[4])
  File "/home/user/projects/anom/main.py", line 33, in train
    engine.fit(datamodule=datamodule, model=model)
  File "/home/user/projects/anom/.venv/lib/python3.11/site-packages/anomalib/engine/engine.py", line 549, in fit
    self.trainer.fit(model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
  File "/home/user/projects/anom/.venv/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py", line 538, in fit
    call._call_and_handle_interrupt(
  File "/home/user/projects/anom/.venv/lib/python3.11/site-packages/lightning/pytorch/trainer/call.py", line 47, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/projects/anom/.venv/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py", line 574, in _fit_impl
    self._run(model, ckpt_path=ckpt_path)
  File "/home/user/projects/anom/.venv/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py", line 943, in _run
    call._call_setup_hook(self)  # allow user to set up LightningModule in accelerator environment
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/projects/anom/.venv/lib/python3.11/site-packages/lightning/pytorch/trainer/call.py", line 102, in _call_setup_hook
    _call_lightning_datamodule_hook(trainer, "setup", stage=fn)
  File "/home/user/projects/anom/.venv/lib/python3.11/site-packages/lightning/pytorch/trainer/call.py", line 189, in _call_lightning_datamodule_hook
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/user/projects/anom/.venv/lib/python3.11/site-packages/anomalib/data/base/datamodule.py", line 138, in setup
    self._setup(stage)
  File "/home/user/projects/anom/.venv/lib/python3.11/site-packages/anomalib/data/image/folder.py", line 446, in _setup
    self.train_data = FolderDataset(
                      ^^^^^^^^^^^^^^
  File "/home/user/projects/anom/.venv/lib/python3.11/site-packages/anomalib/data/image/folder.py", line 261, in __init__
    self.samples = make_folder_dataset(
                   ^^^^^^^^^^^^^^^^^^^^
  File "/home/user/projects/anom/.venv/lib/python3.11/site-packages/anomalib/data/image/folder.py", line 124, in make_folder_dataset
    filename, label = _prepare_files_labels(path, dir_type, extensions)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/projects/anom/.venv/lib/python3.11/site-packages/anomalib/data/utils/path.py", line 74, in _prepare_files_labels
    raise RuntimeError(msg)
RuntimeError: Found 0 DirType.NORMAL images in /home/user/.config/pics/good with extensions ('.jpg', '.jpeg', '.png', '.ppm', '.bmp', '.pgm', '.tif', '.tiff', '.webp')

Code of Conduct

  • I agree to follow this project's Code of Conduct
@terrarier2111
Copy link
Contributor Author

Another consideration i had that this might also be related to hidden files/directories as .config is such a hidden directory

@alexriedel1
Copy link
Contributor

Thi functionality was introduced in 69f922e but I'm not sure if it is even necessary @samet-akcay ?

@terrarier2111
Copy link
Contributor Author

Oh okay, i see, unfortunately for me this is hindering my application, it would be nice to have any way to skip this check or forcefully allow using directories in hidden paths

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants